Oct 30 19:18:12.494945 ip-10-0-143-135 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:18:12.911069 ip-10-0-143-135 kubenswrapper[2547]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:12.911069 ip-10-0-143-135 kubenswrapper[2547]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:18:12.911069 ip-10-0-143-135 kubenswrapper[2547]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:12.911069 ip-10-0-143-135 kubenswrapper[2547]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:12.914152 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914082    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.914152 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914152    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914156    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914160    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914163    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914167    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914172    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914175    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914179    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914182    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914185    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914188    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914190    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914193    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914195    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914198    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914201    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914203    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914205    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914208    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.914215 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914210    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914214    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914218    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914222    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914225    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914234    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914237    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914240    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914243    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914246    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914249    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914252    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914254    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914257    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914259    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914262    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914264    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914267    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914269    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.914640 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914273    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914275    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914278    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914280    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914283    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914285    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914288    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914291    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914294    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914297    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914299    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914302    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914304    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914307    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914310    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914312    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914315    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914317    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914320    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914322    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.915096 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914325    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914328    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914330    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914333    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914336    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914338    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914341    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914343    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914346    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914348    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914351    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914353    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914356    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914358    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914361    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914363    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914366    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914369    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914372    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.915566 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914375    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914378    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914381    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914384    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914386    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914389    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914392    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914394    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914397    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914399    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914402    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914404    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914406    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914409    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914412    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914414    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914417    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914419    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914422    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.916016 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914424    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914427    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914430    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914432    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914434    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914437    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914440    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914442    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.916471 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.914445    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.916935 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916922    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.916935 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916934    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916937    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916941    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916944    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916947    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916950    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916953    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916956    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916959    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916961    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916964    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916967    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916969    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916972    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916974    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916978    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916982    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916985    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916988    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.916993 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916990    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916993    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916996    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.916998    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917001    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917004    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917007    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917009    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917012    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917015    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917017    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917019    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917022    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917024    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917027    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917030    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917032    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917035    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917037    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917040    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.917470 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917043    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917045    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917048    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917050    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917053    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917055    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917057    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917061    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917065    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917067    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917070    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917073    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917075    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917078    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917081    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917084    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917086    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917089    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917091    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917094    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.917986 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917096    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917099    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917102    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917104    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917107    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917109    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917112    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917114    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917117    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917119    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917121    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917124    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917127    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917130    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917132    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917135    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917137    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917140    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917142    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.918455 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917145    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917147    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917150    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917152    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917156    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917159    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917161    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917164    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917167    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917169    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917172    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917174    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917177    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917179    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917182    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917184    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917187    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917190    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917192    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917194    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.918912 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917197    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917200    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917203    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917206    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917208    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917211    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.917213    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918353    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918360    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918363    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918366    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918369    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918371    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918374    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918377    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918379    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918382    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918387    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918391    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.919421 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918394    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918398    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918401    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918404    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918407    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918410    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918413    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918415    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918418    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918420    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918423    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918425    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918428    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918430    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918433    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918436    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918438    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918441    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918443    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.919894 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918446    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918449    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918451    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918454    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918457    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918460    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918464    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918468    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918471    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918474    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918478    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918480    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918483    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918486    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918489    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918492    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918495    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918497    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918500    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918503    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.920340 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918505    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918508    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918510    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918513    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918515    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918518    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918520    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918522    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918525    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918527    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918530    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918532    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918534    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918537    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918540    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918542    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918545    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918548    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918550    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918552    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.920817 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918555    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918558    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918561    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918564    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918567    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918569    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918572    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918574    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918577    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918579    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918582    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918584    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918587    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918589    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918592    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918594    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918597    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918599    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918602    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918604    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.921273 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918607    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918609    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918612    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918614    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918617    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918619    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918622    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918625    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918627    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918630    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918632    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918636    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918639    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918641    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.918644    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919855    2547 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919865    2547 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919871    2547 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919875    2547 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919880    2547 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919883    2547 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:18:12.921764 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919888    2547 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919892    2547 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919895    2547 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919898    2547 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919901    2547 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919904    2547 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919907    2547 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919910    2547 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919913    2547 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919916    2547 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919919    2547 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919922    2547 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919927    2547 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919930    2547 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919933    2547 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919936    2547 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919939    2547 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919943    2547 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919947    2547 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919950    2547 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919953    2547 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919956    2547 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919959    2547 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919962    2547 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:18:12.922279 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919965    2547 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919969    2547 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919973    2547 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919976    2547 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919979    2547 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919982    2547 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919985    2547 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919988    2547 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919993    2547 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919996    2547 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.919999    2547 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920002    2547 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920005    2547 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920009    2547 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920012    2547 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920015    2547 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920018    2547 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920021    2547 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920024    2547 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920027    2547 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920030    2547 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920033    2547 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920036    2547 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920039    2547 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920061    2547 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:18:12.922958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920065    2547 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920068    2547 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920072    2547 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920075    2547 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920078    2547 flags.go:64] FLAG: --help="false"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920081    2547 flags.go:64] FLAG: --hostname-override="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920085    2547 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920088    2547 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920091    2547 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920095    2547 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920098    2547 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920101    2547 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920104    2547 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920108    2547 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920111    2547 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920113    2547 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920117    2547 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920119    2547 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920122    2547 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920125    2547 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920128    2547 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920131    2547 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920134    2547 flags.go:64] FLAG: --lock-file=""
Oct 30 19:18:12.923654 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920137    2547 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920140    2547 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920145    2547 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920150    2547 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920153    2547 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920156    2547 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920159    2547 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920162    2547 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920165    2547 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920168    2547 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920171    2547 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920175    2547 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920179    2547 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920183    2547 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920186    2547 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920189    2547 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920192    2547 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920195    2547 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920198    2547 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920201    2547 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920204    2547 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920211    2547 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920214    2547 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920217    2547 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920220    2547 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:18:12.924282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920222    2547 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920225    2547 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920228    2547 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920231    2547 flags.go:64] FLAG: --port="10250"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920234    2547 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920237    2547 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-034041a3654e951c9"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920240    2547 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920243    2547 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920246    2547 flags.go:64] FLAG: --register-node="true"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920250    2547 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920254    2547 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920256    2547 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920259    2547 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920264    2547 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920267    2547 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920270    2547 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920274    2547 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920277    2547 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920280    2547 flags.go:64] FLAG: --runonce="false"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920283    2547 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920287    2547 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920290    2547 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920293    2547 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920296    2547 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920299    2547 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920302    2547 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:18:12.924960 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920305    2547 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920308    2547 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920310    2547 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920313    2547 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920316    2547 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920319    2547 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920322    2547 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920325    2547 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920330    2547 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920333    2547 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920335    2547 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920340    2547 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920343    2547 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920346    2547 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920349    2547 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920353    2547 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920356    2547 flags.go:64] FLAG: --v="2"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920360    2547 flags.go:64] FLAG: --version="false"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920364    2547 flags.go:64] FLAG: --vmodule=""
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920368    2547 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920371    2547 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920505    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920509    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920512    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.925574 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920515    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920518    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920520    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920523    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920527    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920530    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920534    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920537    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920540    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920543    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920545    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920548    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920550    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920553    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920555    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920558    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920561    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920563    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920566    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.926157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920568    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920571    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920573    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920576    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920578    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920581    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920583    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920586    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920589    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920591    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920594    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920596    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920598    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920601    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920603    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920606    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920609    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920612    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920614    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920617    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.926616 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920619    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920622    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920624    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920627    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920630    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920632    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920635    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920637    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920639    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920643    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920647    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920650    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920653    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920656    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920658    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920661    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920663    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920666    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920668    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.927097 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920671    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920684    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920687    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920690    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920692    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920695    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920697    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920700    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920703    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920705    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920708    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920711    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920714    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920717    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920719    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920722    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920724    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920727    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920729    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920732    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.927587 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920735    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920738    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920740    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920743    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920745    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920748    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920750    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920753    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920755    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920758    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920760    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920763    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920765    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920768    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920770    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920773    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920775    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920778    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920781    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.928078 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920783    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920786    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920788    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920791    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920793    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920796    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.920822    2547 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920869    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920873    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920877    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920880    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920884    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920886    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920893    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920895    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920898    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.928547 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920900    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920903    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920905    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920908    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920910    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920913    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920915    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920918    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920922    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920925    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920927    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920930    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920932    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920935    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920938    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920940    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920943    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920945    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920948    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.929034 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920951    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920953    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920956    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920959    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920962    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920964    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920966    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920969    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920971    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920974    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920978    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920980    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920983    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920986    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920988    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920991    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920993    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.920996    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921000    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.929811 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921004    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921007    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921011    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921014    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921016    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921019    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921022    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921024    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921027    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921029    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921032    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921034    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921037    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921040    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921042    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921045    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921047    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921050    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921053    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921056    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.930491 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921058    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921061    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921063    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921067    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921070    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921073    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921076    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921078    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921081    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921084    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921086    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921089    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921091    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921094    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921097    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921100    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921103    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921105    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921108    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921110    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.931417 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921113    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921115    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921118    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921120    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921123    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921125    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921128    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921130    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921133    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921135    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921139    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921141    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921144    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921147    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921149    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921153    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921155    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921159    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.921161    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.926992    2547 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:18:12.932043 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.927007    2547 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.927021    2547 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.927026    2547 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927081    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927086    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927089    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927092    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927096    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927099    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927102    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927105    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927108    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927111    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927114    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927116    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927119    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927122    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927124    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927127    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.932512 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927130    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927132    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927135    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927137    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927140    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927142    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927145    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927147    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927150    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927152    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927155    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927157    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927160    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927164    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927167    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927170    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927172    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927175    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927178    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927181    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.933051 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927183    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927186    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927188    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927191    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927193    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927196    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927199    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927202    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927204    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927207    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927209    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927212    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927214    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927217    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927220    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927222    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927225    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927227    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927230    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927232    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.933651 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927235    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927245    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927249    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927252    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927255    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927258    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927264    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927268    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927271    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927274    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927276    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927279    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927282    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927284    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927287    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927289    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927292    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927295    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927297    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.934138 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927299    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927302    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927304    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927307    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927309    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927312    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927314    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927317    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927319    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927322    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927324    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927327    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927329    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927332    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927335    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927337    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927339    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927342    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927346    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927350    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.934639 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927353    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927363    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927366    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927368    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927371    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927373    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927376    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927378    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927381    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927383    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927386    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.927413    2547 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927542    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927547    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927550    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927553    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.935147 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927555    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927558    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927560    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927563    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927566    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927568    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927571    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927573    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927576    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927579    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927581    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927583    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927586    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927589    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927592    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927594    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927597    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927600    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927603    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.935601 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927607    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927610    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927613    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927615    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927617    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927621    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927625    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927628    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927630    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927633    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927636    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927638    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927640    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927643    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927645    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927648    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927651    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927653    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927655    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.936056 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927658    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927660    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927663    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927665    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927668    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927670    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927684    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927687    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927690    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927692    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927695    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927698    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927701    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927704    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927706    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927709    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927711    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927714    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927716    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927719    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.936820 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927722    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927724    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927727    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927729    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927732    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927734    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927737    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927739    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927742    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927744    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927747    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927750    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927753    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927755    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927758    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927760    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927763    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927765    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927768    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927771    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.937305 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927773    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927776    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927778    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927781    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927783    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927786    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927789    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927791    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927794    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927796    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927799    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927801    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927804    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927806    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927809    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927811    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927816    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927818    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927821    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927823    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.938015 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927826    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927828    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927831    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927833    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.927856    2547 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927885    2547 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927889    2547 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927892    2547 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927895    2547 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927898    2547 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927901    2547 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927904    2547 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927907    2547 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927910    2547 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927912    2547 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927915    2547 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:12.938492 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927917    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927920    2547 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927923    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927925    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927928    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927930    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927933    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927935    2547 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927938    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927940    2547 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927943    2547 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927945    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927949    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927951    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927954    2547 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927956    2547 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927959    2547 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927961    2547 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927964    2547 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927966    2547 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:12.938919 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927969    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927971    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927974    2547 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927976    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927979    2547 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927981    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927984    2547 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927986    2547 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927989    2547 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927991    2547 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927994    2547 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927996    2547 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.927999    2547 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928001    2547 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928004    2547 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928007    2547 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928009    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928012    2547 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928014    2547 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928017    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:12.939413 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928019    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928022    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928024    2547 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928027    2547 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928030    2547 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928033    2547 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928036    2547 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928038    2547 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928040    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928043    2547 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928047    2547 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928052    2547 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928055    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928058    2547 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928060    2547 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928064    2547 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928068    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928071    2547 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928074    2547 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:12.939893 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928078    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928081    2547 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928084    2547 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928087    2547 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928090    2547 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928092    2547 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928095    2547 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928098    2547 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928100    2547 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928103    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928106    2547 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928108    2547 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928111    2547 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928113    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928116    2547 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928118    2547 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928121    2547 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928125    2547 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928128    2547 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928130    2547 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:12.940331 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928133    2547 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928135    2547 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928138    2547 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928140    2547 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928144    2547 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928147    2547 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928149    2547 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928152    2547 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928154    2547 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928157    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928159    2547 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928162    2547 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928165    2547 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928168    2547 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928170    2547 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:12.928173    2547 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.928221    2547 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.933243    2547 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.934213    2547 server.go:1014] "Starting client certificate rotation"
Oct 30 19:18:12.941067 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.934310    2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:18:12.941879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.934342    2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:18:12.941879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.936736    2547 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:18:12.941879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.941870    2547 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:18:12.956348 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.956332    2547 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:18:12.959417 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.959404    2547 log.go:25] "Validated CRI v1 image API"
Oct 30 19:18:12.960280 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.960258    2547 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:18:12.964522 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.964505    2547 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c8b2ab0d-9680-4eab-9c12-85e7f33af419:/dev/nvme0n1p4 d64950ea-213b-486f-ac74-e8b5fdff87a6:/dev/nvme0n1p3]
Oct 30 19:18:12.964570 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.964523    2547 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 19:18:12.967264 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.967249    2547 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:18:12.969824 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.969722    2547 manager.go:217] Machine: {Timestamp:2026-10-30 19:18:12.96836471 +0000 UTC m=+0.354476243 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23bdb4743ec0a7e3b13a1886f33972 SystemUUID:ec23bdb4-743e-c0a7-e3b1-3a1886f33972 BootID:4e41448b-f5a9-42e1-849a-80b06b335c9d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f4:99:9a:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f4:99:9a:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:98:4c:94:16:52 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:18:12.969824 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.969823    2547 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:18:12.969923 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.969890    2547 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:18:12.972662 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972643    2547 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:18:12.972816 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972663    2547 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-135.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:18:12.972861 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972818    2547 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:18:12.972861 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972826    2547 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:18:12.972861 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972849    2547 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:18:12.972935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.972864    2547 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:18:12.973661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.973651    2547 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:18:12.977178 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.977167    2547 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:18:12.978318 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978308    2547 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:18:12.978349 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978341    2547 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:18:12.978375 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978353    2547 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:18:12.978375 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978371    2547 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:18:12.978439 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978380    2547 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:18:12.978439 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978388    2547 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:18:12.978858 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.978840    2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:18:12.981836 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.981805    2547 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:18:12.986862 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.986847    2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:18:12.987696 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987669    2547 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:18:12.987788 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987708    2547 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:18:12.987880 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987866    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:18:12.987930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987884    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:18:12.987930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987890    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:18:12.987930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987896    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:18:12.987930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987902    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:18:12.987930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.987910    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:18:12.988445 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988432    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:18:12.988480 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988448    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:18:12.988480 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988460    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:18:12.988480 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988471    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:18:12.988608 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988484    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:18:12.988608 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.988497    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:18:12.989066 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.989055    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:18:12.989096 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.989072    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:18:12.990986 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.990964    2547 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-135.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 19:18:12.992921 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.992908    2547 server.go:1263] "Started kubelet"
Oct 30 19:18:12.993022 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.992973    2547 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:18:12.993077 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.993000    2547 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:18:12.993077 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.993063    2547 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:18:12.993640 ip-10-0-143-135 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:18:12.994724 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.994054    2547 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:18:12.994724 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:12.994068    2547 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:18:13.000317 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.000293    2547 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 19:18:13.001591 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.001569    2547 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 19:18:13.001663 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.001586    2547 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-135.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 19:18:13.001928 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.001914    2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:18:13.002501 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.002485    2547 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:18:13.002616 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.001613    2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-135.us-east-2.compute.internal.18e364c97253a8fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-135.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-135.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:18:12.992878844 +0000 UTC m=+0.378990376,LastTimestamp:2026-10-30 19:18:12.992878844 +0000 UTC m=+0.378990376,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-135.us-east-2.compute.internal,}"
Oct 30 19:18:13.002767 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.002750    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.003283 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003262    2547 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:18:13.003283 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003280    2547 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:18:13.003441 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003271    2547 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:18:13.003441 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003356    2547 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:18:13.003441 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003364    2547 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:18:13.003831 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003812    2547 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:18:13.003831 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003833    2547 factory.go:55] Registering systemd factory
Oct 30 19:18:13.003958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.003841    2547 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:18:13.004059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.004042    2547 factory.go:153] Registering CRI-O factory
Oct 30 19:18:13.004059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.004060    2547 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:18:13.004130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.004077    2547 factory.go:103] Registering Raw factory
Oct 30 19:18:13.004130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.004090    2547 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:18:13.004989 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.004977    2547 manager.go:319] Starting recovery of all containers
Oct 30 19:18:13.006085 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.006064    2547 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-135.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 19:18:13.013645 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.013622    2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z6stb"
Oct 30 19:18:13.015986 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.015941    2547 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 19:18:13.016627 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.016610    2547 manager.go:324] Recovery completed
Oct 30 19:18:13.017738 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.017717    2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z6stb"
Oct 30 19:18:13.018273 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.018122    2547 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 19:18:13.021494 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.021479    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.023222 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023205    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.023304 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023245    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.023304 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023259    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.023822 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023806    2547 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:18:13.023822 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023820    2547 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:18:13.023822 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023807    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.023944 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.023839    2547 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:18:13.024575 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.024558    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.024648 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.024589    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.024648 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.024605    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.025835 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.025747    2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-135.us-east-2.compute.internal.18e364c97422a6f9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-135.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-135.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-135.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:18:13.023221497 +0000 UTC m=+0.409333039,LastTimestamp:2026-10-30 19:18:13.023221497 +0000 UTC m=+0.409333039,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-135.us-east-2.compute.internal,}"
Oct 30 19:18:13.025935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.025923    2547 policy_none.go:50] "Start"
Oct 30 19:18:13.025975 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.025942    2547 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:18:13.025975 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.025967    2547 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:18:13.027995 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.027983    2547 policy_none.go:44] "Start"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.065284    2547 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.069950    2547 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.069984    2547 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.069994    2547 server.go:87] "Starting device plugin registration server"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.070209    2547 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.070220    2547 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.070381    2547 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.070470    2547 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.070479    2547 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.070757    2547 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.070789    2547 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.074835    2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.075898    2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.075917    2547 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.075932    2547 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.075961    2547 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 19:18:13.085229 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.079859    2547 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:13.170918 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.170873    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.171573 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.171560    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.171651 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.171585    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.171651 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.171595    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.171651 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.171616    2547 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.176668 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.176652    2547 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"]
Oct 30 19:18:13.176719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.176707    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.177359 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.177346    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.177417 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.177370    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.177417 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.177381    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.181110 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181099    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.181247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181234    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.181283 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181267    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.181666 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181648    2547 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.181778 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.181688    2547 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-135.us-east-2.compute.internal\": node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.181868 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181848    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.181868 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181862    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.181984 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181907    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.181984 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181921    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.181984 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181877    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.181984 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.181961    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.185471 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.185452    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.185556 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.185480    2547 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:13.186240 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.186224    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:13.186331 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.186251    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:13.186331 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.186263    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:13.200043 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.200028    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.204310 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.204291    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.204378 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.204326    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.204378 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.204355    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.206122 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.206095    2547 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.209334 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.209316    2547 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.300888 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.300869    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.305171 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305157    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.305220 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305178    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.305220 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305194    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.305220 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305218    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.305307 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305254    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.305307 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.305295    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.401355 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.401313    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.502050 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.502010    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.509429 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.509412    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.512335 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.512323    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:13.602377 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.602358    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.702773 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.702759    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.803197 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.803152    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.814741 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.814722    2547 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:13.903782 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:13.903763    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:13.934078 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.934060    2547 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 19:18:13.934411 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.934152    2547 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:18:13.934411 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.934192    2547 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:18:13.969407 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:13.969384    2547 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:14.004043 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:14.004020    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:14.022210 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.022181    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:13:13 +0000 UTC" deadline="2028-07-03 05:42:51.737226286 +0000 UTC"
Oct 30 19:18:14.022210 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.022209    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14674h24m37.715020206s"
Oct 30 19:18:14.104325 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:14.104296    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:14.193388 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:14.193361    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51f119163141e2efcccd6dec5fb66e6.slice/crio-666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762 WatchSource:0}: Error finding container 666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762: Status 404 returned error can't find the container with id 666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762
Oct 30 19:18:14.195157 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:14.195127    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c464cbdff5acf5d1b867b786705241.slice/crio-972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5 WatchSource:0}: Error finding container 972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5: Status 404 returned error can't find the container with id 972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5
Oct 30 19:18:14.198612 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.198596    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:18:14.204807 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:14.204788    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:14.305020 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:14.304984    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:14.408758 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:14.408701    2547 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:14.496373 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.496351    2547 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:14.502865 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.502842    2547 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:14.521597 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.521577    2547 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:14.522626 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.522613    2547 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:14.534562 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.534545    2547 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:14.980738 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.980508    2547 apiserver.go:52] "Watching apiserver"
Oct 30 19:18:14.995826 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.995797    2547 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:18:14.996682 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.996651    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-tcmfn","openshift-dns/node-resolver-ln7fq","openshift-network-diagnostics/network-check-target-frfms","kube-system/global-pull-secret-syncer-5qbqv","openshift-multus/multus-additional-cni-plugins-xc95j","openshift-multus/network-metrics-daemon-6752x","openshift-ovn-kubernetes/ovnkube-node-bkpng","kube-system/konnectivity-agent-s2mwq","openshift-image-registry/node-ca-85l8w","openshift-multus/multus-2tv78","openshift-network-operator/iptables-alerter-sdqn8","kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"]
Oct 30 19:18:14.999567 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:14.999540    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.001962 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.001939    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.002976 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.002958    2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 19:18:15.004809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.004526    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t67x5\""
Oct 30 19:18:15.004809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.004767    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.004809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.004767    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:18:15.004809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.004770    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.005064 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.004982    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:18:15.005064 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.005029    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:18:15.005064 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.005032    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.006670 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.006647    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.006809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.006791    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vj9wr\""
Oct 30 19:18:15.006871 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.006801    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.006871 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.006822    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:18:15.007400 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.007380    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.011568 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.011176    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:15.011568 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.011256    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:15.011568 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.011539    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.013422 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013399    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.013538 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013433    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttzl\" (UniqueName: \"kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.013538 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013464    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.013538 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013489    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.013725 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013540    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.013725 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013604    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqkt7\" (UniqueName: \"kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.013725 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013649    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.013729    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013651    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013789    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013815    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013855    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.013887 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013880    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013926    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.013970    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014032    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014077    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014117    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014198 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014150    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014255    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52j5x\" (UniqueName: \"kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014296    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014325    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014355    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014380    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014417    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.014503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.014462    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.019081 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.019058    2547 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:18:15.023150 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.023134    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.025588 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.025301    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.027953 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.027927    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.028053 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.028024    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:15.030639 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.030622    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.033576 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.033335    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.035951 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.035931    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.040736 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.040705    2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x72d7"
Oct 30 19:18:15.050646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.050609    2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x72d7"
Oct 30 19:18:15.083419 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.083371    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerStarted","Data":"972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5"}
Oct 30 19:18:15.085312 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.085237    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"c51f119163141e2efcccd6dec5fb66e6","Type":"ContainerStarted","Data":"666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762"}
Oct 30 19:18:15.105257 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.105235    2547 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:18:15.115283 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115256    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwdtp\" (UniqueName: \"kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.115388 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115294    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.115388 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115321    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.115388 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115346    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.115388 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115376    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115405    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115432    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115458    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115483    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115509    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115541    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115566    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.115604 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115589    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115615    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttzl\" (UniqueName: \"kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115639    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115663    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115687    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115712    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115751    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115774    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115797    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115821    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115844    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115869    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpkh6\" (UniqueName: \"kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115895    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115920    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115945    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.115974    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116000    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116028    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zst7\" (UniqueName: \"kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116053    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116078    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116104    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116132    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116161    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116186    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116210    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116241    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116267    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116294    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116318    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116343    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhjgv\" (UniqueName: \"kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116367    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116393    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116420    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116447    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.116569 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116475    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116501    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116526    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116551    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7vn7\" (UniqueName: \"kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116579    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116604    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116631    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116659    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116684    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116710    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116736    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116775    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116801    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116827    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpvv\" (UniqueName: \"kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116854    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116881    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116905    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116930    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116956    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.116991    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117019    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117052    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117085    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52j5x\" (UniqueName: \"kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117111    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117139    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117165    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117192    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117219    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117245    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117271    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117296    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117330    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117357    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkt7\" (UniqueName: \"kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117396    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.117637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117424    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.118309 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.117619    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.118309 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.118143    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.118309 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.118249    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.118792 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.118771    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.119384 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119368    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:18:15.119495 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119480    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vhzsf\""
Oct 30 19:18:15.119603 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119589    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dk8pw\""
Oct 30 19:18:15.119672 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119603    2547 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:18:15.119891 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119873    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rhqgv\""
Oct 30 19:18:15.119958 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.119937    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:18:15.120027 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.120013    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.120785 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.120768    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.120888 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.120874    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:18:15.121305 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121290    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vd8dh\""
Oct 30 19:18:15.121382 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121332    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.121382 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121370    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.121494 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121448    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:18:15.121494 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121482    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.121595 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121524    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctbj8\""
Oct 30 19:18:15.121645 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121636    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:18:15.121697 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121678    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:18:15.121779 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121742    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-n5j86\""
Oct 30 19:18:15.121898 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121809    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:18:15.121898 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121845    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:18:15.121898 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.121883    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.122131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.122113    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.122194 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.122169    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:18:15.122277 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.122263    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:18:15.122324 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.122308    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:18:15.122367 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.122359    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6rfl7\""
Oct 30 19:18:15.123047 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123023    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.123171 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123151    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.123269 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123252    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.123332 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123307    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123865    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.123982    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124100    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124162    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124181    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124291    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124308    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124368    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124416    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124414    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.124545    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.126774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.125632    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.127442 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.126818    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.128900 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.128877    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttzl\" (UniqueName: \"kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.136104 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.136084    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkt7\" (UniqueName: \"kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.136447 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.136421    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52j5x\" (UniqueName: \"kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.218527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218498    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218540    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218566    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218590    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218611    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218628    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218642    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.218646 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218657    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218735    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218759    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218756    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218783    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218807    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218832    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218835    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218873    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdtp\" (UniqueName: \"kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218900    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218925    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218948    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218976    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219000    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219022    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219063 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219049    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219077    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219102    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219127    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219153    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219179    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219208    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219230    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219238    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219254    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219300    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219337    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219367    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219395    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpkh6\" (UniqueName: \"kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219422    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219449    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219472    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.219827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219559    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zst7\" (UniqueName: \"kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219587    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219600    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219612    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.218874    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219640    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219651    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219609    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219309    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219803    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.220661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.219826    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.221679 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.221627    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:15.221906 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.221883    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.221984 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.221968    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.222059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222027    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.222250 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.222234    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:15.72219819 +0000 UTC m=+3.108309736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:15.222324 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222309    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.222428 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222393    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.222595 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222576    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.222691 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222669    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.222827 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222811    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.222884 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222852    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.222940 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.222930    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223046 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223031    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223123 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223084    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223207 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223192    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223276 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223237    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.223362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223347    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.223442 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.223379    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:15.223496 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.223445    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:15.723421328 +0000 UTC m=+3.109532888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:15.223709 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223688    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.223709 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223701    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223785    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.223879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223828    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.223879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223853    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224023 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223901    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224023 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223923    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjgv\" (UniqueName: \"kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.224023 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.223975    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.224111 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224043    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.224111 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224092    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.224169 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224128    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.224257 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224239    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224292 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224272    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.224345 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224312    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.224431 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224404    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7vn7\" (UniqueName: \"kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.224463 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224451    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224490    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224812    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.224935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224811    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.224935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224872    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.224935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224905    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.225075 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224932    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.225075 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.224953    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.225075 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225037    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225145    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225193    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225203    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225220    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225232    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpvv\" (UniqueName: \"kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.225248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225241    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.225573 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225262    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.225573 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225288    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.225573 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225293    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.225573 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225392    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.225837 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225587    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.225947 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225925    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.226057 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225994    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.226057 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.225996    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.226237 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.226212    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.234838 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.234785    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:15.234838 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.234809    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:15.234838 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.234823    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:15.235041 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.234878    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:15.734861482 +0000 UTC m=+3.120973026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:15.238415 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.236547    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.238513 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.238494    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpvv\" (UniqueName: \"kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.239114 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.239082    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjgv\" (UniqueName: \"kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.239114 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.239098    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zst7\" (UniqueName: \"kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.239258 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.239129    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpkh6\" (UniqueName: \"kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.239391 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.239365    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdtp\" (UniqueName: \"kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.240508 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.240468    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7vn7\" (UniqueName: \"kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.340533 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.340498    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:15.343357 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.343322    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:15.349349 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.349326    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:15.349457 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.349422    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:15.351293 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.351263    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3daf3466_fda2_4bcd_9394_099194ee7c76.slice/crio-de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7 WatchSource:0}: Error finding container de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7: Status 404 returned error can't find the container with id de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7
Oct 30 19:18:15.355826 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.355797    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b937576_a399_42c1_91c3_6dc38e654ce0.slice/crio-57beff71d3f734163ff7576862c4d6a74f4359ea18e06d16c3cdca81829b94ac WatchSource:0}: Error finding container 57beff71d3f734163ff7576862c4d6a74f4359ea18e06d16c3cdca81829b94ac: Status 404 returned error can't find the container with id 57beff71d3f734163ff7576862c4d6a74f4359ea18e06d16c3cdca81829b94ac
Oct 30 19:18:15.362067 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.362031    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07071eae_8f8c_4d40_a3c3_57db59367dfb.slice/crio-9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204 WatchSource:0}: Error finding container 9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204: Status 404 returned error can't find the container with id 9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204
Oct 30 19:18:15.365788 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.365704    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0549cb11_95ca_4c56_bc03_69fe3609450d.slice/crio-7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758 WatchSource:0}: Error finding container 7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758: Status 404 returned error can't find the container with id 7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758
Oct 30 19:18:15.376366 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.376344    2547 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:15.382850 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.382828    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:15.384972 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.384950    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:15.391416 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.391401    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:15.393595 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.393574    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tv78"
Oct 30 19:18:15.394115 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.394092    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1db0ad_000a_4196_a19f_d452c4856568.slice/crio-d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab WatchSource:0}: Error finding container d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab: Status 404 returned error can't find the container with id d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab
Oct 30 19:18:15.395299 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.395263    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:15.396446 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.396418    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067b4348_3847_4370_bd9c_0995f43a8e85.slice/crio-a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef WatchSource:0}: Error finding container a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef: Status 404 returned error can't find the container with id a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef
Oct 30 19:18:15.413119 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:15.413082    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3fe59d_f66e_4dca_9e76_40062110cc31.slice/crio-272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1 WatchSource:0}: Error finding container 272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1: Status 404 returned error can't find the container with id 272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1
Oct 30 19:18:15.729211 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.729130    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:15.729211 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.729170    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:15.729383 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.729294    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:15.729383 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.729313    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:15.729383 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.729351    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:16.729332818 +0000 UTC m=+4.115444342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:15.729383 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.729378    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:16.72935904 +0000 UTC m=+4.115470567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:15.830236 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:15.830179    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:15.830404 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.830369    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:15.830404 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.830386    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:15.830404 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.830396    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:15.830527 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:15.830450    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:16.83043705 +0000 UTC m=+4.216548590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:16.055843 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.055381    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:15 +0000 UTC" deadline="2028-04-24 20:58:52.427048407 +0000 UTC"
Oct 30 19:18:16.056438 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.055871    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13009h40m36.371199413s"
Oct 30 19:18:16.076953 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.076928    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:16.077075 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.077058    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:16.092069 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.092042    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"c51f119163141e2efcccd6dec5fb66e6","Type":"ContainerStarted","Data":"e3479eadf7b36f307a659f5a6aa94bddc967ff69239efa6d4a2ba8e6e751f88b"}
Oct 30 19:18:16.094414 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.094313    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1"}
Oct 30 19:18:16.097541 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.097516    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-85l8w" event={"ID":"5895a27a-6712-4344-ac33-f49f30f5e920","Type":"ContainerStarted","Data":"e24911188dc9ddd41217d73c436600f3e2e6b9e7cb9405d292d647784f5c3e6e"}
Oct 30 19:18:16.099069 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.099027    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ln7fq" event={"ID":"07071eae-8f8c-4d40-a3c3-57db59367dfb","Type":"ContainerStarted","Data":"9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204"}
Oct 30 19:18:16.100937 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.100911    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tcmfn" event={"ID":"8b937576-a399-42c1-91c3-6dc38e654ce0","Type":"ContainerStarted","Data":"57beff71d3f734163ff7576862c4d6a74f4359ea18e06d16c3cdca81829b94ac"}
Oct 30 19:18:16.103027 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.103003    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tv78" event={"ID":"c46ea340-83b0-46fb-ab5e-560f43d8b4e3","Type":"ContainerStarted","Data":"f1cccfe5457131af27e938d5d3f6d2502362c6914f287e692efa4557eb07785c"}
Oct 30 19:18:16.104692 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.104614    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf" event={"ID":"067b4348-3847-4370-bd9c-0995f43a8e85","Type":"ContainerStarted","Data":"a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef"}
Oct 30 19:18:16.109282 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.108496    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal" podStartSLOduration=1.1084843229999999 podStartE2EDuration="1.108484323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:16.108027904 +0000 UTC m=+3.494139448" watchObservedRunningTime="2026-10-30 19:18:16.108484323 +0000 UTC m=+3.494595866"
Oct 30 19:18:16.110405 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.110356    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s2mwq" event={"ID":"dc1db0ad-000a-4196-a19f-d452c4856568","Type":"ContainerStarted","Data":"d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab"}
Oct 30 19:18:16.114986 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.114928    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerStarted","Data":"7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758"}
Oct 30 19:18:16.118290 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.118235    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdqn8" event={"ID":"3daf3466-fda2-4bcd-9394-099194ee7c76","Type":"ContainerStarted","Data":"de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7"}
Oct 30 19:18:16.124059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.123305    2547 generic.go:356] "Generic (PLEG): container finished" podID="93c464cbdff5acf5d1b867b786705241" containerID="bfa6e0ad1ac35bda3f937f1a179cf1776a7bcca2b561870b56d8f20a3049baae" exitCode=0
Oct 30 19:18:16.124059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.123340    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerDied","Data":"bfa6e0ad1ac35bda3f937f1a179cf1776a7bcca2b561870b56d8f20a3049baae"}
Oct 30 19:18:16.420468 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.420435    2547 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.737397    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.737490    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.737685    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.737764    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:18.737744862 +0000 UTC m=+6.123856391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.738262    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:16.738396 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.738314    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:18.738299033 +0000 UTC m=+6.124410575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:16.838831 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:16.838240    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:16.838831 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.838432    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:16.838831 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.838449    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:16.838831 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.838459    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:16.838831 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:16.838505    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:18.838487348 +0000 UTC m=+6.224598886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:17.056383 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:17.056268    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:15 +0000 UTC" deadline="2028-07-27 23:40:26.198801458 +0000 UTC"
Oct 30 19:18:17.056383 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:17.056344    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15268h22m9.142464395s"
Oct 30 19:18:17.084327 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:17.084299    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:17.084481 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:17.084445    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:17.084906 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:17.084884    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:17.085054 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:17.085004    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:17.131393 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:17.131278    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerStarted","Data":"1a8a7f8b6b0fc1546d00e49f9dfb14cf0906f5f4b4b148156a693c6d671b4270"}
Oct 30 19:18:18.076294 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:18.076268    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:18.076797 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.076399    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:18.757652 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:18.757353    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:18.757652 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:18.757485    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:18.757652 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.757501    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:18.757652 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.757579    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:22.757559782 +0000 UTC m=+10.143671332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:18.757652 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.757619    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:18.758058 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.757673    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:22.757656908 +0000 UTC m=+10.143768446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:18.858202 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:18.858156    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:18.858392 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.858335    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:18.858392 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.858359    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:18.858392 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.858371    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:18.858564 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:18.858428    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:22.858410184 +0000 UTC m=+10.244521722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:19.079436 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:19.079353    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:19.079947 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:19.079480    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:19.079947 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:19.079864    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:19.080073 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:19.079967    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:19.146464 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:19.145641    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="2d8e9c4a56eee3065ce1bb0e9abaa30c0697ef48bf967b721a59ac0f56cf0125" exitCode=0
Oct 30 19:18:19.146464 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:19.145697    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"2d8e9c4a56eee3065ce1bb0e9abaa30c0697ef48bf967b721a59ac0f56cf0125"}
Oct 30 19:18:19.169034 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:19.168985    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" podStartSLOduration=4.168968394 podStartE2EDuration="4.168968394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:17.149734679 +0000 UTC m=+4.535846228" watchObservedRunningTime="2026-10-30 19:18:19.168968394 +0000 UTC m=+6.555079940"
Oct 30 19:18:20.077345 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:20.076801    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:20.077345 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:20.076930    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:20.154022 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:20.153226    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s2mwq" event={"ID":"dc1db0ad-000a-4196-a19f-d452c4856568","Type":"ContainerStarted","Data":"34caab3e6b871623851c5c0e1120e874b86864c6a6d5a79acafa94e9f4cd57b9"}
Oct 30 19:18:21.078419 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:21.077862    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:21.078419 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:21.077991    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:21.078930 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:21.078738    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:21.078930 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:21.078854    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:21.160681 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:21.160593    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-85l8w" event={"ID":"5895a27a-6712-4344-ac33-f49f30f5e920","Type":"ContainerStarted","Data":"7676f4f1ad5dbde265240f1a6a8778dd848bf5f10bd596d3a2502a246380eefd"}
Oct 30 19:18:21.179087 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:21.178552    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s2mwq" podStartSLOduration=2.8662261730000003 podStartE2EDuration="7.178532834s" totalImagesPullingTime="4.312306661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:20.171540606 +0000 UTC m=+7.557652154" watchObservedRunningTime="2026-10-30 19:18:21.178532834 +0000 UTC m=+8.564644383"
Oct 30 19:18:22.076611 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:22.076109    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:22.076611 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.076257    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:22.164895 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:22.164797    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf" event={"ID":"067b4348-3847-4370-bd9c-0995f43a8e85","Type":"ContainerStarted","Data":"34424367b7cac97341a3600e583f89ee5232d7c1dfb7957fb81709155e0e7f3a"}
Oct 30 19:18:22.788991 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:22.788939    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:22.788991 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:22.788993    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:22.789235 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.789129    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:22.789235 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.789194    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.789175399 +0000 UTC m=+18.175286940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:22.789650 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.789626    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:22.789742 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.789692    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.789676094 +0000 UTC m=+18.175787635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:22.889916 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:22.889503    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:22.889916 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.889691    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:22.889916 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.889721    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:22.889916 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.889734    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:22.889916 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:22.889801    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:30.889786127 +0000 UTC m=+18.275897673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:23.086735 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:23.086640    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:23.086895 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:23.086818    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:23.087281 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:23.087261    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:23.087435 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:23.087415    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:23.588961 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:23.588937    2547 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:18:24.077591 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.076790    2547 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:18:23.588957198Z","UUID":"f41d024a-67fd-4bc9-bd8d-ad44d51b2fd9","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:18:24.077591 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.076880    2547 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:24.077591 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.076964    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:24.077901 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:24.077806    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:24.078888 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.078265    2547 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:24.078888 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.078514    2547 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:18:24.078888 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.078534    2547 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:18:24.094141 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.094092    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-85l8w" podStartSLOduration=4.770664701 podStartE2EDuration="10.09407638s" totalImagesPullingTime="5.323411679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:21.179381977 +0000 UTC m=+8.565493524" watchObservedRunningTime="2026-10-30 19:18:24.09407638 +0000 UTC m=+11.480187928"
Oct 30 19:18:24.180767 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.172855    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tv78" event={"ID":"c46ea340-83b0-46fb-ab5e-560f43d8b4e3","Type":"ContainerStarted","Data":"08bb7511aee5be340809bac622d1097628fe9eebc1391fc300e209b01f0f178e"}
Oct 30 19:18:24.180767 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.175654    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf" event={"ID":"067b4348-3847-4370-bd9c-0995f43a8e85","Type":"ContainerStarted","Data":"4a00a93a1c37bd565bfc8191d90a38097b5e24dee2b66c47d268093fb00ce2a5"}
Oct 30 19:18:24.184763 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.181606    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerStarted","Data":"c1ef95c281dcc769b69652d7d6858ea3e67acad70ec208b9289d5431cfeab7f1"}
Oct 30 19:18:24.184763 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.183731    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ln7fq" event={"ID":"07071eae-8f8c-4d40-a3c3-57db59367dfb","Type":"ContainerStarted","Data":"87bfd82be66f87465fea7d45f2034526901b8dde26e61a67accbf44284adac7f"}
Oct 30 19:18:24.190314 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:24.190271    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2tv78" podStartSLOduration=2.011515412 podStartE2EDuration="10.190257809s" totalImagesPullingTime="8.178742397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:24.190078011 +0000 UTC m=+11.576189558" watchObservedRunningTime="2026-10-30 19:18:24.190257809 +0000 UTC m=+11.576369355"
Oct 30 19:18:25.076978 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.076895    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:25.077514 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:25.077037    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:25.077514 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.077409    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:25.077617 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:25.077529    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:25.189486 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.189230    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf" event={"ID":"067b4348-3847-4370-bd9c-0995f43a8e85","Type":"ContainerStarted","Data":"81ac933f5358175627082e814f24b28e4c15a36bd4d4b547935f1b18a59d196f"}
Oct 30 19:18:25.191727 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.191678    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="c1ef95c281dcc769b69652d7d6858ea3e67acad70ec208b9289d5431cfeab7f1" exitCode=0
Oct 30 19:18:25.191864 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.191726    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"c1ef95c281dcc769b69652d7d6858ea3e67acad70ec208b9289d5431cfeab7f1"}
Oct 30 19:18:25.209574 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.209532    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf" podStartSLOduration=2.25709557 podStartE2EDuration="11.209521418s" totalImagesPullingTime="8.952425848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:25.209111186 +0000 UTC m=+12.595222733" watchObservedRunningTime="2026-10-30 19:18:25.209521418 +0000 UTC m=+12.595632978"
Oct 30 19:18:25.209785 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:25.209724    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-ln7fq" podStartSLOduration=2.825474823 podStartE2EDuration="11.209717754s" totalImagesPullingTime="8.384242931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:24.229255485 +0000 UTC m=+11.615367032" watchObservedRunningTime="2026-10-30 19:18:25.209717754 +0000 UTC m=+12.595829308"
Oct 30 19:18:26.077026 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:26.076987    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:26.077534 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:26.077109    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:26.197885 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:26.197852    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdqn8" event={"ID":"3daf3466-fda2-4bcd-9394-099194ee7c76","Type":"ContainerStarted","Data":"e634e948fc15b478ae5024267cb6ed4f59c7060503b2acf0d1768d6830f99a2c"}
Oct 30 19:18:27.076913 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.076877    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:27.077075 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.076877    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:27.077075 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:27.077021    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:27.077799 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:27.077096    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:27.201911 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.201874    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tcmfn" event={"ID":"8b937576-a399-42c1-91c3-6dc38e654ce0","Type":"ContainerStarted","Data":"0173e5c0a81808d79be2faa0cbacca89e832262a176ef7ec639398321d1f494f"}
Oct 30 19:18:27.204716 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.204662    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="f969ea909240d724b491abbe1d3ecb7fad5ef221dd508c3a5eeb64c467e4f526" exitCode=0
Oct 30 19:18:27.204848 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.204715    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"f969ea909240d724b491abbe1d3ecb7fad5ef221dd508c3a5eeb64c467e4f526"}
Oct 30 19:18:27.218393 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.218347    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tcmfn" podStartSLOduration=2.404420336 podStartE2EDuration="13.218336753s" totalImagesPullingTime="10.813916417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:27.21830594 +0000 UTC m=+14.604417483" watchObservedRunningTime="2026-10-30 19:18:27.218336753 +0000 UTC m=+14.604448295"
Oct 30 19:18:27.218941 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:27.218907    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdqn8" podStartSLOduration=4.628263118 podStartE2EDuration="13.218897145s" totalImagesPullingTime="8.590634027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:26.213993989 +0000 UTC m=+13.600105532" watchObservedRunningTime="2026-10-30 19:18:27.218897145 +0000 UTC m=+14.605008689"
Oct 30 19:18:28.076846 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:28.076608    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:28.076997 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:28.076951    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:29.076581 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:29.076542    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:29.076581 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:29.076561    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:29.077106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:29.076666    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:29.077106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:29.077078    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:29.211497 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:29.211467    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="90116250eecc250bd5e2426055f2b30d7fc3f26734a790c4369a7d1d084b9d2c" exitCode=0
Oct 30 19:18:29.211650 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:29.211514    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"90116250eecc250bd5e2426055f2b30d7fc3f26734a790c4369a7d1d084b9d2c"}
Oct 30 19:18:30.076718 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:30.076660    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:30.077156 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.076802    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:30.848431 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:30.848184    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:30.848431 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:30.848413    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:30.848667 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.848338    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:30.848667 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.848537    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:30.848667 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.848539    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:46.848517844 +0000 UTC m=+34.234629384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:18:30.848667 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.848598    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret podName:03f7a52b-5c76-43a0-8310-3c56ce2e07d2 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:46.84858251 +0000 UTC m=+34.234694032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret") pod "global-pull-secret-syncer-5qbqv" (UID: "03f7a52b-5c76-43a0-8310-3c56ce2e07d2") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:18:30.949609 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:30.949582    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:30.949775 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.949754    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:18:30.949847 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.949786    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:18:30.949847 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.949799    2547 projected.go:196] Error preparing data for projected volume kube-api-access-7lsws for pod openshift-network-diagnostics/network-check-target-frfms: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:30.949944 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:30.949857    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws podName:46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:46.949838279 +0000 UTC m=+34.335949819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lsws" (UniqueName: "kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws") pod "network-check-target-frfms" (UID: "46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:18:31.077138 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.077115    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:31.077747 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.077145    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:31.077747 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:31.077257    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:31.077747 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:31.077349    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:31.218762 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218736    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"3eea8ca20ae2659ba7b39c070e57eb66ec11a8a32a38a40c4d8630c95805e888"}
Oct 30 19:18:31.218847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218797    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"ff3d42375ab0e7fdba4b29ab061f4fb8d16ab40f075925b793df05ebcfe35e54"}
Oct 30 19:18:31.218847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218841    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"0bf2b8f3cf0950ce6b0f677459a9f4dde36bf606dcd40a67cec95095cb433114"}
Oct 30 19:18:31.218940 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218877    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"a4de9321b112b494cd81e159d9334cf84088084164ae8673a4327d33d4b4c7dd"}
Oct 30 19:18:31.218940 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218905    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"3c09d2ba766390d359b032b9eecaf9388ee2ba44ab7718da8d87bee854bb8d07"}
Oct 30 19:18:31.218940 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.218932    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"972abcda2fefa444c73e5ce1a5b5c5df2ded66b0b3ab23d885e3ae3fa27f3ca5"}
Oct 30 19:18:31.221345 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.221323    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="27cdae3d9cd6550459c4f288a3bf18c51ad27b89556153e87bc12ed63d2152c5" exitCode=0
Oct 30 19:18:31.221419 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:31.221358    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"27cdae3d9cd6550459c4f288a3bf18c51ad27b89556153e87bc12ed63d2152c5"}
Oct 30 19:18:32.076416 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:32.076390    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:32.076530 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:32.076473    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:32.225497 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:32.225471    2547 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="90057d948b25a5834ecf0e48fad1e7e388d4f009aad0b752ddfd36989f4c6c2d" exitCode=0
Oct 30 19:18:32.225852 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:32.225505    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerDied","Data":"90057d948b25a5834ecf0e48fad1e7e388d4f009aad0b752ddfd36989f4c6c2d"}
Oct 30 19:18:33.077070 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:33.077045    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:33.077203 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:33.077134    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:33.077247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:33.077221    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:33.077352 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:33.077324    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:33.229579 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:33.229554    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"903c73b63a34faccefa13c0be4ca520a0299e21b45d321ab60dde40758413bec"}
Oct 30 19:18:33.232630 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:33.232608    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xc95j" event={"ID":"0549cb11-95ca-4c56-bc03-69fe3609450d","Type":"ContainerStarted","Data":"d13b8934a1d1eb80bf7039bb72d90cb9079bf72857a65bdfed9b282be2a5e895"}
Oct 30 19:18:33.256825 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:33.256789    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xc95j" podStartSLOduration=7.78445933 podStartE2EDuration="19.256778012s" totalImagesPullingTime="11.472318682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:33.256228549 +0000 UTC m=+20.642340092" watchObservedRunningTime="2026-10-30 19:18:33.256778012 +0000 UTC m=+20.642889555"
Oct 30 19:18:34.076620 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:34.076588    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:34.076793 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:34.076698    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:35.079155 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.078921    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:35.079155 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.078953    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:35.079621 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:35.079164    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:35.079621 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:35.079231    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:35.239649 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.239625    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" event={"ID":"7b3fe59d-f66e-4dca-9e76-40062110cc31","Type":"ContainerStarted","Data":"c457e6dc9da6779096cdfcc1cc0c398acaa4c1d392efd84d1f6b4ca68f753e39"}
Oct 30 19:18:35.239902 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.239883    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:35.255011 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.254992    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:35.266401 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:35.266364    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng" podStartSLOduration=6.461657641 podStartE2EDuration="21.266354786s" totalImagesPullingTime="14.804697145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:35.265949235 +0000 UTC m=+22.652060778" watchObservedRunningTime="2026-10-30 19:18:35.266354786 +0000 UTC m=+22.652466355"
Oct 30 19:18:36.076591 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.076555    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:36.076772 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:36.076698    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:36.241270 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.241242    2547 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:36.241639 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.241623    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:36.255460 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.255439    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:36.364069 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.364009    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qbqv"]
Oct 30 19:18:36.364209 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.364129    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:36.364269 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:36.364247    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:36.367717 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.367693    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6752x"]
Oct 30 19:18:36.367855 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.367834    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:36.367971 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:36.367948    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:36.368349 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.368328    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frfms"]
Oct 30 19:18:36.368432 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:36.368396    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:36.368491 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:36.368465    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:37.243053 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:37.243026    2547 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:38.076826 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:38.076793    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:38.077016 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:38.076883    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:38.077016 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:38.076894    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:38.077016 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:38.076967    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:38.077016 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:38.076981    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:38.077209 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:38.077057    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:38.245226 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:38.245197    2547 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:40.076835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:40.076806    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:40.077249 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:40.076849    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:40.077249 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:40.076814    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:40.077249 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:40.076908    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:40.077249 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:40.076989    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:40.077249 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:40.077080    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:42.076633 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.076602    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:42.077121 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.076634    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:42.077121 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.076751    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:42.077121 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:42.076752    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6752x" podUID="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c"
Oct 30 19:18:42.077121 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:42.076821    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frfms" podUID="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5"
Oct 30 19:18:42.077121 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:42.076888    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qbqv" podUID="03f7a52b-5c76-43a0-8310-3c56ce2e07d2"
Oct 30 19:18:42.217503 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.217477    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:42.217610 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.217588    2547 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:42.218093 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:42.218069    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:43.409901 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.409755    2547 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeReady"
Oct 30 19:18:43.410241 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.409962    2547 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 19:18:43.443937 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.443918    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9j7fn"]
Oct 30 19:18:43.449639 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.449615    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-67jwx"]
Oct 30 19:18:43.449774 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.449755    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:43.454875 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.454855    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-897tx"]
Oct 30 19:18:43.454993 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.454974    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:43.457344 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.457287    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.458403 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.458380    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:18:43.459194 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.459176    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:43.460248 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.460229    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.461962 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.461872    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"]
Oct 30 19:18:43.462782 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.462758    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.467153 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.467115    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"]
Oct 30 19:18:43.468338 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.468317    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.471608 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.471588    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"]
Oct 30 19:18:43.472024 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.471993    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.473844 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.473824    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 19:18:43.474211 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.474195    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 19:18:43.474600 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.474582    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kzmfw\""
Oct 30 19:18:43.478411 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.478376    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 19:18:43.479579 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.479552    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-c8m4q"]
Oct 30 19:18:43.481138 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.481112    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.486603 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.486266    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d5c44fb6c-bc7gp"]
Oct 30 19:18:43.486723 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.486619    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.493220 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.493199    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"]
Oct 30 19:18:43.493330 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.493315    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.496575 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.496555    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"]
Oct 30 19:18:43.496804 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.496759    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.501170 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.501150    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:18:43.501389 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.501370    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:43.504262 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.504246    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"]
Oct 30 19:18:43.504349 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.504336    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.507027 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.507012    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"]
Oct 30 19:18:43.507131 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.507116    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.509927 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.509912    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"]
Oct 30 19:18:43.510055 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.510040    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:43.512700 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.512669    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9j7fn"]
Oct 30 19:18:43.512781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.512714    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-897tx"]
Oct 30 19:18:43.512781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.512740    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.512781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.512753    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-67jwx"]
Oct 30 19:18:43.512781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.512768    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nmrtr"]
Oct 30 19:18:43.515760 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.515742    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pt2v2"]
Oct 30 19:18:43.515832 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.515797    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.518531 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518516    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"]
Oct 30 19:18:43.518601 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518540    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-c8m4q"]
Oct 30 19:18:43.518601 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518553    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"]
Oct 30 19:18:43.518601 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518573    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:18:43.518601 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518586    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"]
Oct 30 19:18:43.518601 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518599    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518611    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518623    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d5c44fb6c-bc7gp"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518634    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518645    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pt2v2"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518708    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nmrtr"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518714    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518734    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"]
Oct 30 19:18:43.518847 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518839    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"]
Oct 30 19:18:43.519104 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.518871    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"]
Oct 30 19:18:43.546086 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546067    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.546171 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546097    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.546232 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546167    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.546287 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546239    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.546341 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546288    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.546341 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546316    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5t2\" (UniqueName: \"kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.546433 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546342    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.546433 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546367    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.546531 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546430    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.546531 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546511    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.546633 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546544    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.546633 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546601    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmwzz\" (UniqueName: \"kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.546762 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546644    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.546762 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546664    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.546762 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546736    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.546853 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546777    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.546884 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546847    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j76r\" (UniqueName: \"kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.546915 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546888    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnmqf\" (UniqueName: \"kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf\") pod \"kas-connection-checker-685895c7c9-9j7fn\" (UID: \"d976ba7e-0594-4893-9ef9-4332e38a27c8\") " pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:43.546953 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546917    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgxdh\" (UniqueName: \"kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh\") pod \"kas-connection-checker-685895c7c9-897tx\" (UID: \"393c0959-53ac-4bb4-b156-908f228ce12e\") " pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:43.547012 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.546993    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.547083 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547064    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqt99\" (UniqueName: \"kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99\") pod \"kas-connection-checker-685895c7c9-67jwx\" (UID: \"52a1b4f2-94a1-4c37-b905-296e9d8e1513\") " pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:43.547137 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547100    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.547137 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547127    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.547315 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547152    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.547315 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547229    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.547315 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.547292    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tcsb\" (UniqueName: \"kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.560953 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.560936    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561039 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561009    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561069    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561071    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 19:18:43.561209 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561195    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561268 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561255    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561329 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561314    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561379 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561335    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561429 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561385    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561429 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561393    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 19:18:43.561429 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561396    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 19:18:43.561429 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561409    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561584 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561488    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561584 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561505    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561883 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561867    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.561945 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561925    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.561986 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561980    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 19:18:43.562026 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.561989    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.562060 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562003    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 19:18:43.562060 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562014    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 19:18:43.562305 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562290    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.562425 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562330    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.562425 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562344    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.562425 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562344    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:43.562425 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562383    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:18:43.562425 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.562384    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.563159 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.563141    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 19:18:43.563232 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.563176    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:43.566015 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.565838    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 19:18:43.568209 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.568185    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 19:18:43.568344 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.568329    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 19:18:43.568405 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.568368    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 19:18:43.572735 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.572716    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 19:18:43.572818 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.572770    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 19:18:43.572891 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.572840    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 19:18:43.572891 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.572847    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 19:18:43.573011 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.572959    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:18:43.573213 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573195    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l242f\""
Oct 30 19:18:43.573368 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573332    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-sq4pq\""
Oct 30 19:18:43.573475 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573369    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:18:43.573475 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573338    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 19:18:43.573475 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573415    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dwcnh\""
Oct 30 19:18:43.573475 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573444    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573506    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7n7dp\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573197    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z62xn\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573563    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573584    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktbkj\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573603    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573612    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573615    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573624    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t4bhc\""
Oct 30 19:18:43.573719 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573669    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q64cp\""
Oct 30 19:18:43.574044 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573746    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k7ptj\""
Oct 30 19:18:43.574044 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573799    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w7chb\""
Oct 30 19:18:43.574044 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573873    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zxgfv\""
Oct 30 19:18:43.574044 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.573914    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jgwpx\""
Oct 30 19:18:43.648585 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648554    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.648585 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648589    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.648835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648609    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.648835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648628    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.648835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648646    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.648835 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648809    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648846    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5t2\" (UniqueName: \"kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648874    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648900    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648965    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmwzz\" (UniqueName: \"kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.648985    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.649004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649001    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28thn\" (UniqueName: \"kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.649302 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.649013    2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:43.649302 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.649085    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls podName:f02c732c-d967-4721-acf7-76933fd1aba1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.149067731 +0000 UTC m=+31.535179252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-g24tb" (UID: "f02c732c-d967-4721-acf7-76933fd1aba1") : secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:43.649302 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649198    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.649302 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649234    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8g8f\" (UniqueName: \"kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.649302 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649258    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.649571 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649314    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.649571 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649474    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j76r\" (UniqueName: \"kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.649571 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649507    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwwjh\" (UniqueName: \"kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh\") pod \"volume-data-source-validator-58c9f5f46d-g46gd\" (UID: \"b509b764-b249-45ca-8c13-38332fc6c486\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:43.649571 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649529    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.649571 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649556    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnmqf\" (UniqueName: \"kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf\") pod \"kas-connection-checker-685895c7c9-9j7fn\" (UID: \"d976ba7e-0594-4893-9ef9-4332e38a27c8\") " pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:43.649781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649581    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxdh\" (UniqueName: \"kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh\") pod \"kas-connection-checker-685895c7c9-897tx\" (UID: \"393c0959-53ac-4bb4-b156-908f228ce12e\") " pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:43.649781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649605    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.649781 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649649    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.649962 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649824    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.649962 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649890    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.649995    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.650059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650069    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt99\" (UniqueName: \"kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99\") pod \"kas-connection-checker-685895c7c9-67jwx\" (UID: \"52a1b4f2-94a1-4c37-b905-296e9d8e1513\") " pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:43.650186 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650092    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650186 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650160    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650271 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650196    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcsb\" (UniqueName: \"kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.650271 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650218    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.650271 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650251    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.650396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650275    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.650396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650275    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.650396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650351    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.650396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650388    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.650558 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650415    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.650558 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650445    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650558 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650479    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650558 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650504    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.650558 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650537    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5bjf\" (UniqueName: \"kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf\") pod \"network-check-source-6f78d59f99-vkgxb\" (UID: \"62223f92-bc6c-4bcd-b745-9917efe83350\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650563    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650586    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650607    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650631    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650653    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvkld\" (UniqueName: \"kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650703    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.650780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650771    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650806    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650857    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650873    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650883    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.650997    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.651092 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651045    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc24x\" (UniqueName: \"kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651094    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651102    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651167    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.651220    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.651233    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648766b4bf-fs66r: secret "image-registry-tls" not found
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651282    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651304    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651329    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjv7l\" (UniqueName: \"kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.651362 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651356    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.651821 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.651390    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls podName:172146e6-bd62-4407-93fc-19dc16979806 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.15137218 +0000 UTC m=+31.537483704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls") pod "image-registry-648766b4bf-fs66r" (UID: "172146e6-bd62-4407-93fc-19dc16979806") : secret "image-registry-tls" not found
Oct 30 19:18:43.651821 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651414    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.651821 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651472    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.651821 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651492    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.651821 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651562    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.652066 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.651849    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.653359 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.653333    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.653472 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.653334    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.653721 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.653687    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.653721 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.653669    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.653906 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.653786    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.657135 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.657090    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmwzz\" (UniqueName: \"kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.657690 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.657528    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5t2\" (UniqueName: \"kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:43.659184 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.659159    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j76r\" (UniqueName: \"kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.659303 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.659276    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnmqf\" (UniqueName: \"kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf\") pod \"kas-connection-checker-685895c7c9-9j7fn\" (UID: \"d976ba7e-0594-4893-9ef9-4332e38a27c8\") " pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:43.659643 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.659618    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcsb\" (UniqueName: \"kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.659758 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.659657    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.661097 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.660050    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt99\" (UniqueName: \"kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99\") pod \"kas-connection-checker-685895c7c9-67jwx\" (UID: \"52a1b4f2-94a1-4c37-b905-296e9d8e1513\") " pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:43.661097 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.660511    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:43.661097 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.660717    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxdh\" (UniqueName: \"kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh\") pod \"kas-connection-checker-685895c7c9-897tx\" (UID: \"393c0959-53ac-4bb4-b156-908f228ce12e\") " pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:43.752551 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752528    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.752656 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752555    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.752656 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752573    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.752656 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752639    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752658    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28thn\" (UniqueName: \"kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752713    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752738    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8g8f\" (UniqueName: \"kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752776    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwwjh\" (UniqueName: \"kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh\") pod \"volume-data-source-validator-58c9f5f46d-g46gd\" (UID: \"b509b764-b249-45ca-8c13-38332fc6c486\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752801    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.752849 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752831    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752864    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752886    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752915    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752941    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.752981    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753013    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5bjf\" (UniqueName: \"kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf\") pod \"network-check-source-6f78d59f99-vkgxb\" (UID: \"62223f92-bc6c-4bcd-b745-9917efe83350\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.753049    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753068    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753086    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.753130 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.753111    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.25309515 +0000 UTC m=+31.639206684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753046    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753268    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753296    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753302    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvkld\" (UniqueName: \"kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753358    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753389    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753417    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc24x\" (UniqueName: \"kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753452    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753484    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753515    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753539    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753563    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjv7l\" (UniqueName: \"kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753588    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.753598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753593    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753611    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753636    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.753889    2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.753943    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.253928672 +0000 UTC m=+31.640040206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.753949    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.754022    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754065    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:18:43.754247 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754121    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls podName:fe0bce47-ad9b-4cbe-b544-30d75ae93300 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.2541054 +0000 UTC m=+31.640216928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls") pod "dns-default-nmrtr" (UID: "fe0bce47-ad9b-4cbe-b544-30d75ae93300") : secret "dns-default-metrics-tls" not found
Oct 30 19:18:43.754661 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754355    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:43.754661 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754371    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d4cb9956d-hz6xt: secret "image-registry-tls" not found
Oct 30 19:18:43.754661 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754415    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls podName:5348f947-fe2f-4b4d-ae83-2830d5736837 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.254401066 +0000 UTC m=+31.640512602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls") pod "image-registry-5d4cb9956d-hz6xt" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837") : secret "image-registry-tls" not found
Oct 30 19:18:43.754661 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754441    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.254425442 +0000 UTC m=+31.640536988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:43.754661 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.754652    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.754963 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.754804    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.754963 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754901    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 19:18:43.754963 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:43.754941    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls podName:f40f3f83-5aa4-4505-95e5-d8bc7445d5a3 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:44.254927686 +0000 UTC m=+31.641039230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2mnzx" (UID: "f40f3f83-5aa4-4505-95e5-d8bc7445d5a3") : secret "samples-operator-tls" not found
Oct 30 19:18:43.755169 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.755145    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.755357 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.755330    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.756106 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.756086    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.756489 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.756461    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.756809 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.756790    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.756894 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.756875    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.757416 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.757400    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.764717 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.764691    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28thn\" (UniqueName: \"kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:43.765647 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.765619    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8g8f\" (UniqueName: \"kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:43.765760 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.765727    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:43.765826 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.765808    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwwjh\" (UniqueName: \"kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh\") pod \"volume-data-source-validator-58c9f5f46d-g46gd\" (UID: \"b509b764-b249-45ca-8c13-38332fc6c486\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:43.766287 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.766262    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvkld\" (UniqueName: \"kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:43.766743 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.766720    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5bjf\" (UniqueName: \"kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf\") pod \"network-check-source-6f78d59f99-vkgxb\" (UID: \"62223f92-bc6c-4bcd-b745-9917efe83350\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:43.766877 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.766845    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc24x\" (UniqueName: \"kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.767077 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.767060    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.767568 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.767550    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjv7l\" (UniqueName: \"kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:43.767816 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.767800    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:43.786086 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.786066    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:43.794988 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.794971    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:43.814689 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.814658    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:43.820810 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.820777    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:43.831856 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.831349    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:43.838638 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.838207    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:43.857899 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.851831    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:43.866743 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.866580    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:43.884748 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:43.884213    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:44.059076 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.058802    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9j7fn"]
Oct 30 19:18:44.061505 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.061486    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd976ba7e_0594_4893_9ef9_4332e38a27c8.slice/crio-225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607 WatchSource:0}: Error finding container 225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607: Status 404 returned error can't find the container with id 225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607
Oct 30 19:18:44.067164 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.067142    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-67jwx"]
Oct 30 19:18:44.070198 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.070175    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a1b4f2_94a1_4c37_b905_296e9d8e1513.slice/crio-938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7 WatchSource:0}: Error finding container 938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7: Status 404 returned error can't find the container with id 938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7
Oct 30 19:18:44.076608 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.076587    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:44.076726 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.076624    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:44.076726 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.076624    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:44.081160 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.081113    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:18:44.081290 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.081208    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4vgss\""
Oct 30 19:18:44.081290 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.081250    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs9gl\""
Oct 30 19:18:44.081466 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.081294    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-t757m\""
Oct 30 19:18:44.081466 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.081317    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:18:44.097155 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.096769    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393c0959_53ac_4bb4_b156_908f228ce12e.slice/crio-89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71 WatchSource:0}: Error finding container 89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71: Status 404 returned error can't find the container with id 89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71
Oct 30 19:18:44.098959 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.098478    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-897tx"]
Oct 30 19:18:44.130848 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.130790    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"]
Oct 30 19:18:44.132989 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.132962    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98701aa_9692_47f9_be49_076a56bff175.slice/crio-3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2 WatchSource:0}: Error finding container 3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2: Status 404 returned error can't find the container with id 3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.158330    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.158464    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.158646    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.158661    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648766b4bf-fs66r: secret "image-registry-tls" not found
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.158742    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls podName:172146e6-bd62-4407-93fc-19dc16979806 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.15872221 +0000 UTC m=+32.544833749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls") pod "image-registry-648766b4bf-fs66r" (UID: "172146e6-bd62-4407-93fc-19dc16979806") : secret "image-registry-tls" not found
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.159113    2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:44.159191 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.159161    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls podName:f02c732c-d967-4721-acf7-76933fd1aba1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.159147159 +0000 UTC m=+32.545258697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-g24tb" (UID: "f02c732c-d967-4721-acf7-76933fd1aba1") : secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:44.172925 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.169277    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"]
Oct 30 19:18:44.194388 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.193460    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"]
Oct 30 19:18:44.197233 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.197153    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-c8m4q"]
Oct 30 19:18:44.198116 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.198060    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"]
Oct 30 19:18:44.201270 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.201250    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb509b764_b249_45ca_8c13_38332fc6c486.slice/crio-8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7 WatchSource:0}: Error finding container 8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7: Status 404 returned error can't find the container with id 8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7
Oct 30 19:18:44.203021 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.202869    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda74375_ee6c_4daa_a5dd_74d0df935567.slice/crio-52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968 WatchSource:0}: Error finding container 52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968: Status 404 returned error can't find the container with id 52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968
Oct 30 19:18:44.204519 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.204107    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2c3f45_829b_4784_b1da_5997daf80443.slice/crio-0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376 WatchSource:0}: Error finding container 0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376: Status 404 returned error can't find the container with id 0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376
Oct 30 19:18:44.242465 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.242441    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pt2v2"]
Oct 30 19:18:44.246459 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.246432    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"]
Oct 30 19:18:44.248902 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:44.248880    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62223f92_bc6c_4bcd_b745_9917efe83350.slice/crio-1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7 WatchSource:0}: Error finding container 1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7: Status 404 returned error can't find the container with id 1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7
Oct 30 19:18:44.258492 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.258467    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" event={"ID":"cf2c3f45-829b-4784-b1da-5997daf80443","Type":"ContainerStarted","Data":"0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376"}
Oct 30 19:18:44.259011 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.258989    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:44.259094 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259038    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:44.259094 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259065    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:44.259195 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259099    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:18:44.259195 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259130    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:44.259195 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259152    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls podName:fe0bce47-ad9b-4cbe-b544-30d75ae93300 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.25913287 +0000 UTC m=+32.645244395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls") pod "dns-default-nmrtr" (UID: "fe0bce47-ad9b-4cbe-b544-30d75ae93300") : secret "dns-default-metrics-tls" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259192    2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259206    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259204    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259235    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.259224085 +0000 UTC m=+32.645335610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259263    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.259249464 +0000 UTC m=+32.645360988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259265    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259281    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259296    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d4cb9956d-hz6xt: secret "image-registry-tls" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259303    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259311    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls podName:f40f3f83-5aa4-4505-95e5-d8bc7445d5a3 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.259298678 +0000 UTC m=+32.645410217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2mnzx" (UID: "f40f3f83-5aa4-4505-95e5-d8bc7445d5a3") : secret "samples-operator-tls" not found
Oct 30 19:18:44.259346 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259334    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls podName:5348f947-fe2f-4b4d-ae83-2830d5736837 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.259322836 +0000 UTC m=+32.645434376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls") pod "image-registry-5d4cb9956d-hz6xt" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837") : secret "image-registry-tls" not found
Oct 30 19:18:44.259736 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:44.259376    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:45.259365089 +0000 UTC m=+32.645476632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:44.259736 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.259548    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p" event={"ID":"a98701aa-9692-47f9-be49-076a56bff175","Type":"ContainerStarted","Data":"3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2"}
Oct 30 19:18:44.260874 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.260856    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9j7fn" event={"ID":"d976ba7e-0594-4893-9ef9-4332e38a27c8","Type":"ContainerStarted","Data":"8f1b5c8218bfc3ec1633cc6d8d39aa7a5476d2c7faa951e88799975074e4c4ce"}
Oct 30 19:18:44.260954 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.260891    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9j7fn" event={"ID":"d976ba7e-0594-4893-9ef9-4332e38a27c8","Type":"ContainerStarted","Data":"225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607"}
Oct 30 19:18:44.261901 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.261880    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pt2v2" event={"ID":"cbec4229-cce9-465d-8ff4-c01460c9a1b9","Type":"ContainerStarted","Data":"c520205c1a7dfd4185a5e5363a58c232c13e5ebde327c224e24ebfe5b2f26b74"}
Oct 30 19:18:44.263002 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.262983    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-c8m4q" event={"ID":"cda74375-ee6c-4daa-a5dd-74d0df935567","Type":"ContainerStarted","Data":"52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968"}
Oct 30 19:18:44.264252 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.264230    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-897tx" event={"ID":"393c0959-53ac-4bb4-b156-908f228ce12e","Type":"ContainerStarted","Data":"08a3314606b718ddbf52d07676aded49655895bc6253fd09138a5eecca6169a0"}
Oct 30 19:18:44.264325 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.264271    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-897tx" event={"ID":"393c0959-53ac-4bb4-b156-908f228ce12e","Type":"ContainerStarted","Data":"89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71"}
Oct 30 19:18:44.265780 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.265759    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-67jwx" event={"ID":"52a1b4f2-94a1-4c37-b905-296e9d8e1513","Type":"ContainerStarted","Data":"d47af81ca9e7c199ce8d15644531f4f0e9231ff097a0152d63ddb6eb95f32b85"}
Oct 30 19:18:44.265861 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.265797    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-67jwx" event={"ID":"52a1b4f2-94a1-4c37-b905-296e9d8e1513","Type":"ContainerStarted","Data":"938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7"}
Oct 30 19:18:44.266883 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.266862    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb" event={"ID":"62223f92-bc6c-4bcd-b745-9917efe83350","Type":"ContainerStarted","Data":"1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7"}
Oct 30 19:18:44.267966 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.267919    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd" event={"ID":"b509b764-b249-45ca-8c13-38332fc6c486","Type":"ContainerStarted","Data":"8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7"}
Oct 30 19:18:44.268935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.268917    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2" event={"ID":"4b3670eb-d511-4036-bb90-5fed8a4320b9","Type":"ContainerStarted","Data":"b9c11e9813920a3cb3ca3ff7c03d3c3cee6dc391201036fdbcf502e8591ca251"}
Oct 30 19:18:44.281392 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.281351    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-9j7fn" podStartSLOduration=380.281336832 podStartE2EDuration="6m20.281336832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:44.281276887 +0000 UTC m=+31.667388431" watchObservedRunningTime="2026-10-30 19:18:44.281336832 +0000 UTC m=+31.667448377"
Oct 30 19:18:44.302213 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.302179    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-67jwx" podStartSLOduration=380.302169159 podStartE2EDuration="6m20.302169159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:44.301857076 +0000 UTC m=+31.687968618" watchObservedRunningTime="2026-10-30 19:18:44.302169159 +0000 UTC m=+31.688280700"
Oct 30 19:18:44.326137 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.326100    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-897tx" podStartSLOduration=380.326090934 podStartE2EDuration="6m20.326090934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:44.325251201 +0000 UTC m=+31.711362744" watchObservedRunningTime="2026-10-30 19:18:44.326090934 +0000 UTC m=+31.712202477"
Oct 30 19:18:44.394647 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.394625    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:44.394832 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.394820    2547 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:44.404882 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:44.404862    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:45.170548 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.170483    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.170608    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.170794    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.170810    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648766b4bf-fs66r: secret "image-registry-tls" not found
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.170840    2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.170868    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls podName:172146e6-bd62-4407-93fc-19dc16979806 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.170850436 +0000 UTC m=+34.556961981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls") pod "image-registry-648766b4bf-fs66r" (UID: "172146e6-bd62-4407-93fc-19dc16979806") : secret "image-registry-tls" not found
Oct 30 19:18:45.171042 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.170884    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls podName:f02c732c-d967-4721-acf7-76933fd1aba1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.170875808 +0000 UTC m=+34.556987329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-g24tb" (UID: "f02c732c-d967-4721-acf7-76933fd1aba1") : secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:45.271848 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.271803    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:45.272008 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.271856    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:45.272008 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.271917    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:45.272008 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.271956    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:45.272008 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.271990    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:45.272162 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:45.272093    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:45.272258 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272222    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:45.272294 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272283    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272266439 +0000 UTC m=+34.658377976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:45.272368 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272353    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 19:18:45.272454 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272395    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls podName:f40f3f83-5aa4-4505-95e5-d8bc7445d5a3 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272383504 +0000 UTC m=+34.658495043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2mnzx" (UID: "f40f3f83-5aa4-4505-95e5-d8bc7445d5a3") : secret "samples-operator-tls" not found
Oct 30 19:18:45.272507 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272461    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272451761 +0000 UTC m=+34.658563296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:45.272507 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272521    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:18:45.272653 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272551    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls podName:fe0bce47-ad9b-4cbe-b544-30d75ae93300 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272540555 +0000 UTC m=+34.658652090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls") pod "dns-default-nmrtr" (UID: "fe0bce47-ad9b-4cbe-b544-30d75ae93300") : secret "dns-default-metrics-tls" not found
Oct 30 19:18:45.272653 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272612    2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:45.272653 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272642    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272631919 +0000 UTC m=+34.658743454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:45.272825 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272726    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:45.272825 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272737    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d4cb9956d-hz6xt: secret "image-registry-tls" not found
Oct 30 19:18:45.272825 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:45.272767    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls podName:5348f947-fe2f-4b4d-ae83-2830d5736837 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:47.272757239 +0000 UTC m=+34.658868775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls") pod "image-registry-5d4cb9956d-hz6xt" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837") : secret "image-registry-tls" not found
Oct 30 19:18:46.296438 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.296381    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd" event={"ID":"b509b764-b249-45ca-8c13-38332fc6c486","Type":"ContainerStarted","Data":"01521f348d6fa1b6d69d4bb4be6d539581479c01ef2be5bea714e48f45953758"}
Oct 30 19:18:46.314004 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.312388    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd" podStartSLOduration=354.440177957 podStartE2EDuration="5m56.31237141s" totalImagesPullingTime="1.872193453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:46.312144987 +0000 UTC m=+33.698256531" watchObservedRunningTime="2026-10-30 19:18:46.31237141 +0000 UTC m=+33.698482954"
Oct 30 19:18:46.890956 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.890917    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:46.891115 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.890969    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:46.891529 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:46.891507    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 19:18:46.891634 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:46.891600    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:19:18.891568452 +0000 UTC m=+66.277679975 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : secret "metrics-daemon-secret" not found
Oct 30 19:18:46.897126 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.897102    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:46.992894 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.992414    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:46.995635 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:46.995584    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:47.093698 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.093612    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:47.104212 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.104147    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.201242    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.201385    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.201608    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.201631    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648766b4bf-fs66r: secret "image-registry-tls" not found
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.201799    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls podName:172146e6-bd62-4407-93fc-19dc16979806 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.201776352 +0000 UTC m=+38.587887892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls") pod "image-registry-648766b4bf-fs66r" (UID: "172146e6-bd62-4407-93fc-19dc16979806") : secret "image-registry-tls" not found
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.201729    2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:47.204577 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.201884    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls podName:f02c732c-d967-4721-acf7-76933fd1aba1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.201870421 +0000 UTC m=+38.587981943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-g24tb" (UID: "f02c732c-d967-4721-acf7-76933fd1aba1") : secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.303754    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.303859    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.303894    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.303907    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.303922    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d4cb9956d-hz6xt: secret "image-registry-tls" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.303927    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.303978    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls podName:5348f947-fe2f-4b4d-ae83-2830d5736837 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.303959784 +0000 UTC m=+38.690071327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls") pod "image-registry-5d4cb9956d-hz6xt" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837") : secret "image-registry-tls" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.304009    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304011    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304021    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.304009804 +0000 UTC m=+38.690121344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304047    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.304038047 +0000 UTC m=+38.690149569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.304075    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304089    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 19:18:47.304147 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304121    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls podName:f40f3f83-5aa4-4505-95e5-d8bc7445d5a3 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.304111543 +0000 UTC m=+38.690223065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2mnzx" (UID: "f40f3f83-5aa4-4505-95e5-d8bc7445d5a3") : secret "samples-operator-tls" not found
Oct 30 19:18:47.305220 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304213    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:18:47.305220 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304271    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls podName:fe0bce47-ad9b-4cbe-b544-30d75ae93300 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.304254587 +0000 UTC m=+38.690366122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls") pod "dns-default-nmrtr" (UID: "fe0bce47-ad9b-4cbe-b544-30d75ae93300") : secret "dns-default-metrics-tls" not found
Oct 30 19:18:47.305220 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304302    2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:47.305220 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:47.304348    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.304335752 +0000 UTC m=+38.690447276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:47.422981 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.422952    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frfms"]
Oct 30 19:18:47.426388 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:47.426341    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46297d6c_6db8_4e9d_9e74_8bc1a53fa7e5.slice/crio-d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337 WatchSource:0}: Error finding container d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337: Status 404 returned error can't find the container with id d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337
Oct 30 19:18:47.449332 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:47.449307    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qbqv"]
Oct 30 19:18:47.456881 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:47.456854    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f7a52b_5c76_43a0_8310_3c56ce2e07d2.slice/crio-1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e WatchSource:0}: Error finding container 1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e: Status 404 returned error can't find the container with id 1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e
Oct 30 19:18:48.312038 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:48.311956    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qbqv" event={"ID":"03f7a52b-5c76-43a0-8310-3c56ce2e07d2","Type":"ContainerStarted","Data":"1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e"}
Oct 30 19:18:48.313879 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:48.313799    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frfms" event={"ID":"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5","Type":"ContainerStarted","Data":"d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337"}
Oct 30 19:18:49.323139 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.322333    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/0.log"
Oct 30 19:18:49.323139 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.322374    2547 generic.go:356] "Generic (PLEG): container finished" podID="cf2c3f45-829b-4784-b1da-5997daf80443" containerID="cf2498ad0c05f7a9f3278bc32a415135f07033e3d2a0b9aca9bd33c4be1962e4" exitCode=255
Oct 30 19:18:49.323139 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.322444    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" event={"ID":"cf2c3f45-829b-4784-b1da-5997daf80443","Type":"ContainerDied","Data":"cf2498ad0c05f7a9f3278bc32a415135f07033e3d2a0b9aca9bd33c4be1962e4"}
Oct 30 19:18:49.323139 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.322798    2547 scope.go:122] "RemoveContainer" containerID="cf2498ad0c05f7a9f3278bc32a415135f07033e3d2a0b9aca9bd33c4be1962e4"
Oct 30 19:18:49.327979 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.327520    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p" event={"ID":"a98701aa-9692-47f9-be49-076a56bff175","Type":"ContainerStarted","Data":"d06ecc7c78f584180cb8cbfc443aba8afe39b4cba6b7b2ac2ae2f46a42e965ad"}
Oct 30 19:18:49.332320 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.331721    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-c8m4q" event={"ID":"cda74375-ee6c-4daa-a5dd-74d0df935567","Type":"ContainerStarted","Data":"6d3aa2c33b1965aef8c039e2c9276c124fda472d60ac8955883bea3976a2108b"}
Oct 30 19:18:49.336109 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.335738    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2" event={"ID":"4b3670eb-d511-4036-bb90-5fed8a4320b9","Type":"ContainerStarted","Data":"bea5f2825ea788abdd2b2cce4e80e1997a008e826aaffbca92755a161e5f0963"}
Oct 30 19:18:49.369239 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.368482    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2" podStartSLOduration=354.591312052 podStartE2EDuration="5m59.36846767s" totalImagesPullingTime="4.777155618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:49.367215198 +0000 UTC m=+36.753326742" watchObservedRunningTime="2026-10-30 19:18:49.36846767 +0000 UTC m=+36.754579216"
Oct 30 19:18:49.390135 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.390095    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p" podStartSLOduration=354.553320465 podStartE2EDuration="5m59.390085345s" totalImagesPullingTime="4.83676488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:49.382759751 +0000 UTC m=+36.768871305" watchObservedRunningTime="2026-10-30 19:18:49.390085345 +0000 UTC m=+36.776196887"
Oct 30 19:18:49.412664 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.412619    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-c8m4q" podStartSLOduration=354.658057419 podStartE2EDuration="5m59.412603818s" totalImagesPullingTime="4.754546399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:49.410827072 +0000 UTC m=+36.796938617" watchObservedRunningTime="2026-10-30 19:18:49.412603818 +0000 UTC m=+36.798715364"
Oct 30 19:18:49.632488 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.632419    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b6rzj"]
Oct 30 19:18:49.640353 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.640330    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:18:49.640598 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.640581    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.650182 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.649934    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.651072 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.651057    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:18:49.651446 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.651431    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:18:49.662993 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.662729    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6rzj"]
Oct 30 19:18:49.662993 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.662961    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkcxf\""
Oct 30 19:18:49.665696 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.665656    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833119    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833164    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833193    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833226    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833256    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833271    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpmd5\" (UniqueName: \"kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833293    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833322    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.833527 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.833350    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.933966    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934035    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934082    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934136    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934183    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934206    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmd5\" (UniqueName: \"kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934248    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934295    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934342    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.934784    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:49.934868    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:49.934926    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:50.434906709 +0000 UTC m=+37.821018234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : configmap "kube-rbac-proxy" not found
Oct 30 19:18:49.935796 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:49.935791    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:49.936551 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:49.935852    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:50.435833072 +0000 UTC m=+37.821944605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:49.936551 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.936171    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.936551 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.936259    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.936551 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.936542    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.937107 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.937061    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.943268 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.943247    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"]
Oct 30 19:18:49.953010 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.952983    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmd5\" (UniqueName: \"kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:49.954303 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.954273    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:49.976766 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.975543    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"]
Oct 30 19:18:49.976941 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.976597    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:49.994117 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.993996    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bjlqm\""
Oct 30 19:18:49.995965 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.995350    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"]
Oct 30 19:18:49.995965 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.995393    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"]
Oct 30 19:18:49.995965 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:49.995481    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:50.004779 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.004749    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:50.004881 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.004845    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 19:18:50.008486 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.008466    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:50.137217 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.137186    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrglb\" (UniqueName: \"kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb\") pod \"migrator-66df5d496b-k2nfn\" (UID: \"3f60d3c9-0f82-438d-95f9-ee13814e718b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:50.137217 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.137226    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stvfx\" (UniqueName: \"kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx\") pod \"migrator-66df5d496b-sd6pb\" (UID: \"cc012785-f079-4824-b224-044c52d1b12f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:50.238200 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.238168    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrglb\" (UniqueName: \"kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb\") pod \"migrator-66df5d496b-k2nfn\" (UID: \"3f60d3c9-0f82-438d-95f9-ee13814e718b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:50.238369 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.238214    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stvfx\" (UniqueName: \"kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx\") pod \"migrator-66df5d496b-sd6pb\" (UID: \"cc012785-f079-4824-b224-044c52d1b12f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:50.247990 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.247964    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrglb\" (UniqueName: \"kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb\") pod \"migrator-66df5d496b-k2nfn\" (UID: \"3f60d3c9-0f82-438d-95f9-ee13814e718b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:50.249396 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.249369    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stvfx\" (UniqueName: \"kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx\") pod \"migrator-66df5d496b-sd6pb\" (UID: \"cc012785-f079-4824-b224-044c52d1b12f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:50.307944 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.307917    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:18:50.320929 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.320904    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:50.339800 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.338199    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:50.342272 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.341643    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pt2v2" event={"ID":"cbec4229-cce9-465d-8ff4-c01460c9a1b9","Type":"ContainerStarted","Data":"493ec8e466e6e7580f62faa8211d9a885472df2fbd342bbaddde216ef58abceb"}
Oct 30 19:18:50.345170 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.345085    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" event={"ID":"438e3014-0622-4bfc-b7a0-7da9fc14472b","Type":"ContainerStarted","Data":"93e5d87d97d2c039538170021c792b1fe73900d1f714ffc89a4da842fcc65b5f"}
Oct 30 19:18:50.349298 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.348725    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:50.349298 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.349243    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/0.log"
Oct 30 19:18:50.349298 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.349272    2547 generic.go:356] "Generic (PLEG): container finished" podID="cf2c3f45-829b-4784-b1da-5997daf80443" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf" exitCode=255
Oct 30 19:18:50.349886 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.349869    2547 scope.go:122] "RemoveContainer" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf"
Oct 30 19:18:50.350250 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.350229    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" event={"ID":"cf2c3f45-829b-4784-b1da-5997daf80443","Type":"ContainerDied","Data":"23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf"}
Oct 30 19:18:50.350342 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.350273    2547 scope.go:122] "RemoveContainer" containerID="cf2498ad0c05f7a9f3278bc32a415135f07033e3d2a0b9aca9bd33c4be1962e4"
Oct 30 19:18:50.350892 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:50.350865    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b7c9l_openshift-console-operator(cf2c3f45-829b-4784-b1da-5997daf80443)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" podUID="cf2c3f45-829b-4784-b1da-5997daf80443"
Oct 30 19:18:50.363874 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.363408    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pt2v2" podStartSLOduration=1.9484152670000001 podStartE2EDuration="7.363391646s" totalImagesPullingTime="5.414976379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.360635461 +0000 UTC m=+37.746747004" watchObservedRunningTime="2026-10-30 19:18:50.363391646 +0000 UTC m=+37.749503191"
Oct 30 19:18:50.441825 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.441232    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:50.441825 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.441291    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:50.441825 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:50.441744    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:18:50.441825 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:50.441817    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.441796572 +0000 UTC m=+38.827908093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : configmap "kube-rbac-proxy" not found
Oct 30 19:18:50.442088 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:50.441899    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:50.442088 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:50.441938    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:51.441925357 +0000 UTC m=+38.828036879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:50.628334 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.628311    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"]
Oct 30 19:18:50.630215 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.630187    2547 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2f76w"]
Oct 30 19:18:50.630622 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:50.630598    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc012785_f079_4824_b224_044c52d1b12f.slice/crio-e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721 WatchSource:0}: Error finding container e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721: Status 404 returned error can't find the container with id e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721
Oct 30 19:18:50.650353 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.650309    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2f76w"]
Oct 30 19:18:50.650443 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.650421    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.661294 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.661158    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 19:18:50.661294 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.661258    2547 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mgmzn\""
Oct 30 19:18:50.680659 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.680573    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"]
Oct 30 19:18:50.686182 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:50.686159    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f60d3c9_0f82_438d_95f9_ee13814e718b.slice/crio-e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17 WatchSource:0}: Error finding container e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17: Status 404 returned error can't find the container with id e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17
Oct 30 19:18:50.720217 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.720091    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 19:18:50.720217 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.720174    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 19:18:50.720368 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.720299    2547 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 19:18:50.846791 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.846731    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbjd\" (UniqueName: \"kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.846952 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.846933    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.847059 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.847042    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.947810 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.947775    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.947945 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.947874    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.947999 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.947951    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbjd\" (UniqueName: \"kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.948640 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.948620    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.950276 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.950255    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.956609 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.956587    2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbjd\" (UniqueName: \"kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:50.982808 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:50.982786    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:51.251322 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.251287    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:51.251467 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.251424    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:51.251528 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.251461    2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:51.251588 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.251536    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls podName:f02c732c-d967-4721-acf7-76933fd1aba1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.251515209 +0000 UTC m=+46.637626744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-g24tb" (UID: "f02c732c-d967-4721-acf7-76933fd1aba1") : secret "cluster-monitoring-operator-tls" not found
Oct 30 19:18:51.251648 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.251578    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:51.251648 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.251603    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648766b4bf-fs66r: secret "image-registry-tls" not found
Oct 30 19:18:51.251648 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.251640    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls podName:172146e6-bd62-4407-93fc-19dc16979806 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.251629385 +0000 UTC m=+46.637740906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls") pod "image-registry-648766b4bf-fs66r" (UID: "172146e6-bd62-4407-93fc-19dc16979806") : secret "image-registry-tls" not found
Oct 30 19:18:51.296238 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.296211    2547 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2f76w"]
Oct 30 19:18:51.300363 ip-10-0-143-135 kubenswrapper[2547]: W1030 19:18:51.300334    2547 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91892ae8_06fc_4bb6_a183_c3a1c9440084.slice/crio-63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d WatchSource:0}: Error finding container 63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d: Status 404 returned error can't find the container with id 63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d
Oct 30 19:18:51.352322 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352297    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352338    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352361    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352404    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352440    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.352472    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.352619    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 19:18:51.352637 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.352630    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d4cb9956d-hz6xt: secret "image-registry-tls" not found
Oct 30 19:18:51.353001 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.352668    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls podName:5348f947-fe2f-4b4d-ae83-2830d5736837 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.352657009 +0000 UTC m=+46.738768542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls") pod "image-registry-5d4cb9956d-hz6xt" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837") : secret "image-registry-tls" not found
Oct 30 19:18:51.353001 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.352974    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:51.353106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353026    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.353013631 +0000 UTC m=+46.739125165 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:51.353106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353039    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.353033781 +0000 UTC m=+46.739145304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:51.353106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353057    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:18:51.353106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353085    2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:51.353106 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353098    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 19:18:51.353363 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353103    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls podName:fe0bce47-ad9b-4cbe-b544-30d75ae93300 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.353087463 +0000 UTC m=+46.739199052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls") pod "dns-default-nmrtr" (UID: "fe0bce47-ad9b-4cbe-b544-30d75ae93300") : secret "dns-default-metrics-tls" not found
Oct 30 19:18:51.353363 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353159    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.353143814 +0000 UTC m=+46.739255339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:51.353363 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.353175    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls podName:f40f3f83-5aa4-4505-95e5-d8bc7445d5a3 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.353164984 +0000 UTC m=+46.739276509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2mnzx" (UID: "f40f3f83-5aa4-4505-95e5-d8bc7445d5a3") : secret "samples-operator-tls" not found
Oct 30 19:18:51.363404 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.363372    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frfms" event={"ID":"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5","Type":"ContainerStarted","Data":"af06b22bf5990e7b8f63f43e3f9d3315040bff662491574358f68321b358114a"}
Oct 30 19:18:51.364051 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.364020    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:51.366179 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.366155    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w" event={"ID":"91892ae8-06fc-4bb6-a183-c3a1c9440084","Type":"ContainerStarted","Data":"63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d"}
Oct 30 19:18:51.369263 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.369242    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" event={"ID":"438e3014-0622-4bfc-b7a0-7da9fc14472b","Type":"ContainerStarted","Data":"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"}
Oct 30 19:18:51.371521 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.371495    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb" event={"ID":"62223f92-bc6c-4bcd-b745-9917efe83350","Type":"ContainerStarted","Data":"f6302264168176fc5eea7d1ca0fefb17c3c37518ff17756aba01b7208fbeafd8"}
Oct 30 19:18:51.373199 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.373175    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn" event={"ID":"3f60d3c9-0f82-438d-95f9-ee13814e718b","Type":"ContainerStarted","Data":"e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17"}
Oct 30 19:18:51.380227 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.380202    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb" event={"ID":"cc012785-f079-4824-b224-044c52d1b12f","Type":"ContainerStarted","Data":"e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721"}
Oct 30 19:18:51.382265 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.382221    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-frfms" podStartSLOduration=33.832336928 podStartE2EDuration="37.382209807s" totalImagesPullingTime="3.549872879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:51.381120471 +0000 UTC m=+38.767232013" watchObservedRunningTime="2026-10-30 19:18:51.382209807 +0000 UTC m=+38.768321364"
Oct 30 19:18:51.384714 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.384663    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:51.390432 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.385319    2547 scope.go:122] "RemoveContainer" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf"
Oct 30 19:18:51.390432 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.385504    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b7c9l_openshift-console-operator(cf2c3f45-829b-4784-b1da-5997daf80443)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" podUID="cf2c3f45-829b-4784-b1da-5997daf80443"
Oct 30 19:18:51.398620 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.398392    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb" podStartSLOduration=351.107402789 podStartE2EDuration="5m57.39835611s" totalImagesPullingTime="6.290953321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:51.39738779 +0000 UTC m=+38.783499334" watchObservedRunningTime="2026-10-30 19:18:51.39835611 +0000 UTC m=+38.784467654"
Oct 30 19:18:51.417407 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.417363    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" podStartSLOduration=2.417346404 podStartE2EDuration="2.417346404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:51.417211895 +0000 UTC m=+38.803323442" watchObservedRunningTime="2026-10-30 19:18:51.417346404 +0000 UTC m=+38.803457951"
Oct 30 19:18:51.453195 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.453161    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:51.453385 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:51.453208    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:51.453445 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.453408    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:51.453508 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.453467    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:53.453449213 +0000 UTC m=+40.839560749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:51.453508 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.453574    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:18:51.453744 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:51.453631    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:53.45361489 +0000 UTC m=+40.839726430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : configmap "kube-rbac-proxy" not found
Oct 30 19:18:52.397311 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:52.397233    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w" event={"ID":"91892ae8-06fc-4bb6-a183-c3a1c9440084","Type":"ContainerStarted","Data":"f0aa77f84d3038bd755caca70e6066ba1226885d14d401364de251fc02e77ac2"}
Oct 30 19:18:52.416832 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:52.416777    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w" podStartSLOduration=2.416759287 podStartE2EDuration="2.416759287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:52.414976178 +0000 UTC m=+39.801087722" watchObservedRunningTime="2026-10-30 19:18:52.416759287 +0000 UTC m=+39.802870829"
Oct 30 19:18:53.401961 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.401922    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn" event={"ID":"3f60d3c9-0f82-438d-95f9-ee13814e718b","Type":"ContainerStarted","Data":"b754ab717f7a6a778b4d22f5e1f548ae46c4b8a616c28a805554517ea1f62338"}
Oct 30 19:18:53.403831 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.403803    2547 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb" event={"ID":"cc012785-f079-4824-b224-044c52d1b12f","Type":"ContainerStarted","Data":"280967ec95660974239604c236d68d545a34db7bf50ba6d72fe1f0555deec97b"}
Oct 30 19:18:53.417830 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.417781    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn" podStartSLOduration=2.395707399 podStartE2EDuration="4.417766691s" totalImagesPullingTime="2.022059292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:53.417160528 +0000 UTC m=+40.803272074" watchObservedRunningTime="2026-10-30 19:18:53.417766691 +0000 UTC m=+40.803878235"
Oct 30 19:18:53.433805 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.433767    2547 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb" podStartSLOduration=2.366484258 podStartE2EDuration="4.433755712s" totalImagesPullingTime="2.067271454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:53.43244005 +0000 UTC m=+40.818551593" watchObservedRunningTime="2026-10-30 19:18:53.433755712 +0000 UTC m=+40.819867255"
Oct 30 19:18:53.483935 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.483904    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:53.484076 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.483965    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:53.484250 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:53.484230    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:53.484328 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:53.484291    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.484276419 +0000 UTC m=+44.870387940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:53.484805 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:53.484784    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:18:53.484887 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:53.484842    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.484825894 +0000 UTC m=+44.870937421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : configmap "kube-rbac-proxy" not found
Oct 30 19:18:53.839379 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.839347    2547 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:53.839542 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.839407    2547 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:53.839785 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:53.839770    2547 scope.go:122] "RemoveContainer" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf"
Oct 30 19:18:53.839953 ip-10-0-143-135 kubenswrapper[2547]: E1030 19:18:53.839935    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b7c9l_openshift-console-operator(cf2c3f45-829b-4784-b1da-5997daf80443)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" podUID="cf2c3f45-829b-4784-b1da-5997daf80443"
Oct 30 19:18:54.773094 ip-10-0-143-135 kubenswrapper[2547]: I1030 19:18:54.773073    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ln7fq_07071eae-8f8c-4d40-a3c3-57db59367dfb/dns-node-resolver/0.log"
Oct 30 19:18:54.959534 ip-10-0-143-135 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 19:18:54.979034 ip-10-0-143-135 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 19:18:54.979204 ip-10-0-143-135 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 19:18:54.979259 ip-10-0-143-135 systemd[1]: kubelet.service: Consumed 3.014s CPU time.
Oct 30 19:18:54.992265 ip-10-0-143-135 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:18:55.091565 ip-10-0-143-135 kubenswrapper[5033]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:55.091565 ip-10-0-143-135 kubenswrapper[5033]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:18:55.091565 ip-10-0-143-135 kubenswrapper[5033]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:55.091565 ip-10-0-143-135 kubenswrapper[5033]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:18:55.093649 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093578    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.093649 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093650    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093655    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093658    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093660    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093663    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093666    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093669    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093684    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093688    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093690    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093693    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093697    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093699    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093702    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093704    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093708    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093711    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093714    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093717    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.093743 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093719    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093722    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093724    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093728    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093730    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093733    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093736    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093738    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093741    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093744    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093746    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093749    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093752    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093755    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093757    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093761    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093766    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093769    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093772    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.094279 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093775    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093777    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093780    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093782    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093785    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093788    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093790    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093793    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093795    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093798    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093802    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093805    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093808    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093811    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093814    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093817    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093820    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093823    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093826    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093828    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.095078 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093831    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093834    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093836    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093839    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093842    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093844    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093847    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093850    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093852    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093855    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093858    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093860    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093862    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093865    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093867    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093870    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093873    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093876    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093878    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093881    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.095613 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093885    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093887    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093890    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093895    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093899    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093902    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093905    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093908    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093910    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093913    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093916    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093918    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093920    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093923    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093925    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093928    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093930    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093933    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093935    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.096135 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093938    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093941    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093943    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093946    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093949    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093952    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093954    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.093957    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095791    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095804    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095807    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095811    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095815    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095819    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095821    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095825    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095828    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095831    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095833    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.096627 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095836    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095838    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095841    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095844    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095846    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095849    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095852    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095855    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095857    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095861    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095863    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095866    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095868    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095871    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095874    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095876    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095879    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095881    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095886    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.097132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095891    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095894    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095897    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095900    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095903    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095920    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095925    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095928    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095931    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095933    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095936    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095939    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095941    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095944    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095948    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095951    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095954    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095956    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095960    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095962    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.097600 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095966    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095969    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095971    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095974    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095976    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095979    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095982    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095985    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095987    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095989    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095992    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095994    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.095997    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096000    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096003    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096006    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096009    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096011    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096014    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.098123 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096017    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096020    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096023    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096025    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096028    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096031    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096033    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096036    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096038    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096040    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096043    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096046    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096048    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096051    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096054    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096057    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096059    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096062    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096064    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096067    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.098620 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096069    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096072    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096074    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096077    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096080    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096082    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096085    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096087    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096091    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096093    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096096    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096099    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096101    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096104    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096107    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096110    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096113    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096600    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096607    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096610    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.099136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096613    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096616    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096619    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096621    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096624    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096627    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096630    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096633    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096635    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096638    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096641    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096644    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096647    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096649    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096653    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096656    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096658    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096661    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096663    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096666    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.099615 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096669    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096686    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096690    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096693    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096695    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096698    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096700    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096703    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096706    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096709    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096711    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096714    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096716    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096719    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096721    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096724    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096726    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096729    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096731    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.100158 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096734    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096740    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096743    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096745    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096748    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096750    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096753    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096755    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096758    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096760    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096763    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096766    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096769    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096771    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096774    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096776    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096779    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096782    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096784    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096787    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.100720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096789    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096792    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096794    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096797    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096801    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096805    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096808    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096811    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096813    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096817    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096819    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096822    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096825    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096829    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096832    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096835    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096837    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096840    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096843    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.101232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096845    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096848    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096850    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096853    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096855    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096858    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096861    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096863    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096867    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096870    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096873    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096876    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096878    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096881    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096884    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096886    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096889    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096891    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096894    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.101720 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096897    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096899    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096903    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096905    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096908    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.096910    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.096996    5033 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097005    5033 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097011    5033 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097016    5033 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097020    5033 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097024    5033 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097028    5033 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097032    5033 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097035    5033 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097039    5033 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097042    5033 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097045    5033 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097048    5033 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097051    5033 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097054    5033 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097057    5033 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097060    5033 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:18:55.102187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097063    5033 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097068    5033 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097071    5033 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097074    5033 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097078    5033 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097081    5033 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097085    5033 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097088    5033 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097091    5033 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097095    5033 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097098    5033 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097101    5033 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097104    5033 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097107    5033 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097110    5033 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097114    5033 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097118    5033 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097121    5033 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097125    5033 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097128    5033 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097131    5033 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097135    5033 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097139    5033 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097142    5033 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:18:55.102766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097145    5033 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097148    5033 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097152    5033 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097155    5033 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097158    5033 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097161    5033 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097164    5033 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097167    5033 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097170    5033 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097173    5033 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097176    5033 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097179    5033 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097182    5033 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097188    5033 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097191    5033 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097194    5033 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097197    5033 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097201    5033 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097204    5033 flags.go:64] FLAG: --help="false"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097207    5033 flags.go:64] FLAG: --hostname-override="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097210    5033 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097213    5033 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097216    5033 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:18:55.103355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097220    5033 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097224    5033 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097227    5033 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097230    5033 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097235    5033 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097237    5033 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097241    5033 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097244    5033 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097247    5033 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097249    5033 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097252    5033 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097255    5033 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097258    5033 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097261    5033 flags.go:64] FLAG: --lock-file=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097264    5033 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097267    5033 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097270    5033 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097276    5033 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097279    5033 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097282    5033 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097285    5033 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097288    5033 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097293    5033 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097296    5033 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097299    5033 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:18:55.104025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097304    5033 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097307    5033 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097311    5033 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097314    5033 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097317    5033 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097320    5033 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097323    5033 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097326    5033 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097329    5033 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097332    5033 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097338    5033 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097342    5033 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097345    5033 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097348    5033 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097351    5033 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097354    5033 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097357    5033 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097360    5033 flags.go:64] FLAG: --port="10250"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097363    5033 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097366    5033 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-034041a3654e951c9"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097369    5033 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097372    5033 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097375    5033 flags.go:64] FLAG: --register-node="true"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097378    5033 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097382    5033 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097385    5033 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:18:55.104699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097388    5033 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097390    5033 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097397    5033 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097401    5033 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097405    5033 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097408    5033 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097410    5033 flags.go:64] FLAG: --runonce="false"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097413    5033 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097416    5033 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097419    5033 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097423    5033 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097425    5033 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097429    5033 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097432    5033 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097435    5033 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097438    5033 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097441    5033 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097443    5033 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097446    5033 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097450    5033 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097453    5033 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097456    5033 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097461    5033 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097464    5033 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097466    5033 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:18:55.105349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097470    5033 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097473    5033 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097476    5033 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097479    5033 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097482    5033 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097485    5033 flags.go:64] FLAG: --v="2"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097489    5033 flags.go:64] FLAG: --version="false"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097493    5033 flags.go:64] FLAG: --vmodule=""
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097497    5033 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097500    5033 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097642    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097646    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097651    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097654    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097657    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097660    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097662    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097665    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097667    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097670    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097687    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097690    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.105979 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097693    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097696    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097698    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097701    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097704    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097707    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097710    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097712    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097715    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097718    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097720    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097723    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097726    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097729    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097731    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097734    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097737    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097739    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097742    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097744    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.106865 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097748    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097751    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097755    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097757    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097760    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097763    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097765    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097768    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097770    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097773    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097775    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097778    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097781    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097783    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097786    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097788    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097791    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097794    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097797    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.107430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097799    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097802    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097804    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097807    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097810    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097812    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097815    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097817    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097820    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097822    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097825    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097827    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097830    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097834    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097837    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097841    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097846    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097849    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097853    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.107993 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097856    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097859    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097861    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097864    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097867    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097869    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097872    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097874    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097877    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097880    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097882    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097885    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097887    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097890    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097893    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097897    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097901    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097903    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097906    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097909    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.108536 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097911    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097914    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097917    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097919    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097922    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097924    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097929    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097932    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097936    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097938    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097941    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097944    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097946    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097949    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097951    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.097954    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.109189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.097979    5033 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098015    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098020    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098023    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098026    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098029    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098031    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098034    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098037    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098040    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098043    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098046    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098048    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098051    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098053    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098056    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098059    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098061    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098064    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098066    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.109846 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098069    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098071    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098076    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098078    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098081    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098085    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098087    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098090    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098093    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098096    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098098    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098101    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098103    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098106    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098109    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098111    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098114    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098116    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098119    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.110642 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098121    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098124    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098127    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098129    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098133    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098136    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098139    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098141    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098144    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098146    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098149    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098152    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098154    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098157    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098160    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098164    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098167    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098169    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098174    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.111163 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098176    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098179    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098182    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098184    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098187    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098190    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098192    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098195    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098197    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098200    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098203    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098206    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098208    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098211    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098215    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098219    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098221    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098224    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098228    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.111625 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098231    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098234    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098236    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098239    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098241    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098244    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098246    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098249    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098252    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098256    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098258    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098261    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098265    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098269    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098272    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098275    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098278    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098281    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098283    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098286    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.112095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098289    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098292    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098295    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098297    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098300    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098302    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098305    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098307    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098310    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.098312    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.101731    5033 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.101745    5033 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.101763    5033 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.101767    5033 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101820    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101825    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101828    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101831    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101834    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.112589 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101836    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101839    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101842    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101845    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101847    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101850    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101852    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101855    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101857    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101860    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101863    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101866    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101868    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101871    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101873    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101876    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101879    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101881    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101884    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101887    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.113116 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101890    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101892    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101895    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101899    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101904    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101908    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101911    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101914    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101917    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101919    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101922    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101925    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101928    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101930    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101933    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101936    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101940    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101944    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101948    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.113593 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101952    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101956    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101960    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101965    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101969    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101973    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101977    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101979    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101982    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101985    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101988    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101992    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.101997    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102000    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102004    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102007    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102009    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102013    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102015    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.114109 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102019    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102021    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102024    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102027    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102030    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102032    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102035    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102038    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102040    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102043    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102046    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102048    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102051    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102053    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102056    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102059    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102061    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102064    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102067    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102069    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.114564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102072    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102074    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102077    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102079    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102083    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102085    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102088    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102091    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102093    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102096    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102098    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102101    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102104    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102106    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102109    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102112    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102114    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102117    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102119    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102122    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.115062 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102124    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102127    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102129    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.102157    5033 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102294    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102298    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102301    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102304    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102307    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102310    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102312    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102315    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102318    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102321    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102324    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102326    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.115541 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102329    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102332    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102334    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102337    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102339    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102342    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102344    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102348    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102352    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102355    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102358    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102361    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102364    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102366    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102369    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102371    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102374    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102377    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102379    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.115933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102382    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102385    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102387    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102390    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102392    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102395    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102397    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102400    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102403    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102406    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102408    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102411    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102413    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102416    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102418    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102421    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102424    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102426    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102429    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.116466 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102432    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102434    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102437    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102439    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102442    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102445    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102447    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102450    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102452    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102455    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102458    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102461    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102463    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102466    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102468    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102471    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102474    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102476    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102479    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102481    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.116935 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102484    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102487    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102489    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102492    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102494    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102497    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102499    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102503    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102507    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102510    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102513    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102516    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102519    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102521    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102524    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102527    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102529    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102532    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102537    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.117453 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102540    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102543    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102546    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102548    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102551    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102554    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102556    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102559    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102562    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102565    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102567    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102570    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102572    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102575    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102578    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102580    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.117941 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102583    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.102606    5033 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102639    5033 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102643    5033 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102646    5033 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102649    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102652    5033 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102654    5033 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102657    5033 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102660    5033 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102662    5033 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102665    5033 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102668    5033 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102670    5033 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102690    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102695    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:18:55.118330 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102700    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102703    5033 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102706    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102709    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102711    5033 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102714    5033 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102719    5033 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102722    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102725    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102728    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102731    5033 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102733    5033 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102736    5033 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102739    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102741    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102744    5033 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102746    5033 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102749    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102752    5033 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:18:55.118734 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102754    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102757    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102759    5033 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102762    5033 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102765    5033 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102767    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102770    5033 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102773    5033 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102775    5033 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102778    5033 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102781    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102784    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102786    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102789    5033 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102793    5033 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102795    5033 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102798    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102801    5033 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102803    5033 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:18:55.119232 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102807    5033 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102809    5033 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102812    5033 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102814    5033 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102817    5033 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102819    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102822    5033 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102824    5033 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102827    5033 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102829    5033 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102832    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102835    5033 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102837    5033 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102840    5033 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102842    5033 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102845    5033 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102847    5033 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102850    5033 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102852    5033 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102855    5033 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:18:55.119706 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102857    5033 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102860    5033 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102863    5033 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102865    5033 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102868    5033 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102871    5033 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102875    5033 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102879    5033 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102883    5033 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102886    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102889    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102892    5033 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102894    5033 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102897    5033 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102899    5033 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102902    5033 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102904    5033 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102907    5033 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102910    5033 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:18:55.120217 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102912    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102915    5033 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102917    5033 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102920    5033 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102923    5033 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102925    5033 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102929    5033 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102931    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102934    5033 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102936    5033 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102939    5033 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102941    5033 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102944    5033 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102947    5033 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:55.102950    5033 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.102996    5033 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.103930    5033 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.103986    5033 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.104159    5033 server.go:1014] "Starting client certificate rotation"
Oct 30 19:18:55.120669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.104298    5033 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.104345    5033 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:13:13 +0000 UTC" deadline="2028-05-09 04:50:23.744662631 +0000 UTC"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.104370    5033 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13353h31m28.640294803s"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.104595    5033 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.105560    5033 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.107057    5033 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.108712    5033 log.go:25] "Validated CRI v1 image API"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.109536    5033 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:18:55.121164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.112413    5033 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c8b2ab0d-9680-4eab-9c12-85e7f33af419:/dev/nvme0n1p4 d64950ea-213b-486f-ac74-e8b5fdff87a6:/dev/nvme0n1p3]
Oct 30 19:18:55.121395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.112430    5033 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d/userdata/shm major:0 minor:359 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93e5d87d97d2c039538170021c792b1fe73900d1f714ffc89a4da842fcc65b5f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93e5d87d97d2c039538170021c792b1fe73900d1f714ffc89a4da842fcc65b5f/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5/userdata/shm major:0 minor:49 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b9c11e9813920a3cb3ca3ff7c03d3c3cee6dc391201036fdbcf502e8591ca251/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b9c11e9813920a3cb3ca3ff7c03d3c3cee6dc391201036fdbcf502e8591ca251/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c520205c1a7dfd4185a5e5363a58c232c13e5ebde327c224e24ebfe5b2f26b74/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c520205c1a7dfd4185a5e5363a58c232c13e5ebde327c224e24ebfe5b2f26b74/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e24911188dc9ddd41217d73c436600f3e2e6b9e7cb9405d292d647784f5c3e6e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e24911188dc9ddd41217d73c436600f3e2e6b9e7cb9405d292d647784f5c3e6e/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17/userdata/shm major:0 minor:339 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1cccfe5457131af27e938d5d3f6d2502362c6914f287e692efa4557eb07785c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1cccfe5457131af27e938d5d3f6d2502362c6914f287e692efa4557eb07785c/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03f7a52b-5c76-43a0-8310-3c56ce2e07d2/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/03f7a52b-5c76-43a0-8310-3c56ce2e07d2/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0549cb11-95ca-4c56-bc03-69fe3609450d/volumes/kubernetes.io~projected/kube-api-access-tqkt7:{mountpoint:/var/lib/kubelet/pods/0549cb11-95ca-4c56-bc03-69fe3609450d/volumes/kubernetes.io~projected/kube-api-access-tqkt7 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/067b4348-3847-4370-bd9c-0995f43a8e85/volumes/kubernetes.io~projected/kube-api-access-5kpvv:{mountpoint:/var/lib/kubelet/pods/067b4348-3847-4370-bd9c-0995f43a8e85/volumes/kubernetes.io~projected/kube-api-access-5kpvv major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07071eae-8f8c-4d40-a3c3-57db59367dfb/volumes/kubernetes.io~projected/kube-api-access-hhjgv:{mountpoint:/var/lib/kubelet/pods/07071eae-8f8c-4d40-a3c3-57db59367dfb/volumes/kubernetes.io~projected/kube-api-access-hhjgv major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12011250-d65c-4fa7-9529-bd4731595e41/volumes/kubernetes.io~projected/kube-api-access-tpmd5:{mountpoint:/var/lib/kubelet/pods/12011250-d65c-4fa7-9529-bd4731595e41/volumes/kubernetes.io~projected/kube-api-access-tpmd5 major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/kube-api-access-zglrx:{mountpoint:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/kube-api-access-zglrx major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/393c0959-53ac-4bb4-b156-908f228ce12e/volumes/kubernetes.io~projected/kube-api-access-lgxdh:{mountpoint:/var/lib/kubelet/pods/393c0959-53ac-4bb4-b156-908f228ce12e/volumes/kubernetes.io~projected/kube-api-access-lgxdh major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3daf3466-fda2-4bcd-9394-099194ee7c76/volumes/kubernetes.io~projected/kube-api-access-52j5x:{mountpoint:/var/lib/kubelet/pods/3daf3466-fda2-4bcd-9394-099194ee7c76/volumes/kubernetes.io~projected/kube-api-access-52j5x major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f60d3c9-0f82-438d-95f9-ee13814e718b/volumes/kubernetes.io~projected/kube-api-access-vrglb:{mountpoint:/var/lib/kubelet/pods/3f60d3c9-0f82-438d-95f9-ee13814e718b/volumes/kubernetes.io~projected/kube-api-access-vrglb major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/438e3014-0622-4bfc-b7a0-7da9fc14472b/volumes/kubernetes.io~projected/kube-api-access-2vdrl:{mountpoint:/var/lib/kubelet/pods/438e3014-0622-4bfc-b7a0-7da9fc14472b/volumes/kubernetes.io~projected/kube-api-access-2vdrl major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5/volumes/kubernetes.io~projected/kube-api-access-7lsws:{mountpoint:/var/lib/kubelet/pods/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5/volumes/kubernetes.io~projected/kube-api-access-7lsws major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~projected/kube-api-access-gmwzz:{mountpoint:/var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~projected/kube-api-access-gmwzz major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~secret/serving-cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52a1b4f2-94a1-4c37-b905-296e9d8e1513/volumes/kubernetes.io~projected/kube-api-access-cqt99:{mountpoint:/var/lib/kubelet/pods/52a1b4f2-94a1-4c37-b905-296e9d8e1513/volumes/kubernetes.io~projected/kube-api-access-cqt99 major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/kube-api-access-m6ngh:{mountpoint:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/kube-api-access-m6ngh major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5895a27a-6712-4344-ac33-f49f30f5e920/volumes/kubernetes.io~projected/kube-api-access-l7vn7:{mountpoint:/var/lib/kubelet/pods/5895a27a-6712-4344-ac33-f49f30f5e920/volumes/kubernetes.io~projected/kube-api-access-l7vn7 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c/volumes/kubernetes.io~projected/kube-api-access-5zst7:{mountpoint:/var/lib/kubelet/pods/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c/volumes/kubernetes.io~projected/kube-api-access-5zst7 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62223f92-bc6c-4bcd-b745-9917efe83350/volumes/kubernetes.io~projected/kube-api-access-h5bjf:{mountpoint:/var/lib/kubelet/pods/62223f92-bc6c-4bcd-b745-9917efe83350/volumes/kubernetes.io~projected/kube-api-access-h5bjf major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~projected/kube-api-access-28thn:{mountpoint:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~projected/kube-api-access-28thn major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/default-certificate major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/stats-auth major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~projected/kube-api-access-xpkh6:{mountpoint:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~projected/kube-api-access-xpkh6 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~projected/kube-api-access-lttzl:{mountpoint:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~projected/kube-api-access-lttzl major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~projected/kube-api-access-brbjd:{mountpoint:/var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~projected/kube-api-access-brbjd major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~secret/signing-key major:0 minor:354 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~projected/kube-api-access-9j76r:{mountpoint:/var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~projected/kube-api-access-9j76r major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b509b764-b249-45ca-8c13-38332fc6c486/volumes/kubernetes.io~projected/kube-api-access-wwwjh:{mountpoint:/var/lib/kubelet/pods/b509b764-b249-45ca-8c13-38332fc6c486/volumes/kubernetes.io~projected/kube-api-access-wwwjh major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c46ea340-83b0-46fb-ab5e-560f43d8b4e3/volumes/kubernetes.io~projected/kube-api-access-pwdtp:{mountpoint:/var/lib/kubelet/pods/c46ea340-83b0-46fb-ab5e-560f43d8b4e3/volumes/kubernetes.io~projected/kube-api-access-pwdtp major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~projected/kube-api-access-f8g8f:{mountpoint:/var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~projected/kube-api-access-f8g8f major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~secret/cert major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc012785-f079-4824-b224-044c52d1b12f/volumes/kubernetes.io~projected/kube-api-access-stvfx:{mountpoint:/var/lib/kubelet/pods/cc012785-f079-4824-b224-044c52d1b12f/volumes/kubernetes.io~projected/kube-api-access-stvfx major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~projected/kube-api-access-4tcsb:{mountpoint:/var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~projected/kube-api-access-4tcsb major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~projected/kube-api-access-cc24x:{mountpoint:/var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~projected/kube-api-access-cc24x major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d976ba7e-0594-4893-9ef9-4332e38a27c8/volumes/kubernetes.io~projected/kube-api-access-jnmqf:{mountpoint:/var/lib/kubelet/pods/d976ba7e-0594-4893-9ef9-4332e38a27c8/volumes/kubernetes.io~projected/kube-api-access-jnmqf major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc1db0ad-000a-4196-a19f-d452c4856568/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/dc1db0ad-000a-4196-a19f-d452c4856568/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f02c732c-d967-4721-acf7-76933fd1aba1/volumes/kubernetes.io~projected/kube-api-access-wj5t2:{mountpoint:/var/lib/kubelet/pods/f02c732c-d967-4721-acf7-76933fd1aba1/volumes/kubernetes.io~projected/kube-api-access-wj5t2 major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3/volumes/kubernetes.io~projected/kube-api-access-sjv7l:{mountpoint:/var/lib/kubelet/pods/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3/volumes/kubernetes.io~projected/kube-api-access-sjv7l major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe0bce47-ad9b-4cbe-b544-30d75ae93300/volumes/kubernetes.io~projected/kube-api-access-tvkld:{mountpoint:/var/lib/kubelet/pods/fe0bce47-ad9b-4cbe-b544-30d75ae93300/volumes/kubernetes.io~projected/kube-api-access-tvkld major:0 minor:220 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/3d701e099da36c87f2de20c2a97fcca2a8cfc7e67991e46161140495abde4994/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c83d3f3d8a01511b9c2208798c3a4d64b96c5c3a09a3322d0ed91326d780b972/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/8525c9689bfff9d76451beb0ae0d1c2ea4193eb30978823f0a543d4d7e99c4a5/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/ec9f74b4b037492408a6c9a5c69045438acda56c313958d342df73901a9225ad/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/6548946bdebdcc706f63c0d526d9eac00c982b52e2e23b90f39d952ce96ef9f6/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/49b5e4fb91f6519a91f34e82f090e4c9e459c96d0573b149760473e563b2981d/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2ff9669d7580b474ca5e7ffd6426ac463b50f41074e9705397919211d99a55f8/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/babb0f53608dae2ae12e9a6450498d533d993d2c57a43754ad6cfdaabe718412/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/d43af9dee1913d65aa2fa449b92c14e150d796909b247465da0787c37ae3d0f4/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/7bfe326c1a010ac96d13e19988129cf2152f043c001d864a9c99e407bcfeb178/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/2edf2ea21fda3d39d717a5c3629b95e71a7f1a61012ca1185c736d0b378d497a/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/60798803e97eb29d02c862f4fcabb36f004fb90b3c39b9318beda43a34c93187/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/52c69c391cd3eaba5403ff11257d26e291903f54a1dadd50a39d4dcfa358122d/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/86f141d1b0b1ef98a6378f59b5a3a8e1745c70a8ead2f12bdb5241103611d25e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/fdb7f63912aa37670effaf2ca703e736faee1205173791f7e384bf9c8737866f/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/cebab29a8c73839ebb7d8da050e6a7c61e2d2f6e134b4064dc20a0b4c6b9546d/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/bc73ac6cd5fddb79ea974e45d0375c50045f2909c6320a84a9225a0fbd57ca2d/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5585ea511eebd3f7ef6652239406f452b4e63f5ba315de554a9d03777ef606ea/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/701b5b5accfedcfe47ea24f65537a3e0f72b92493d4839e2864cd0fa5c459bb6/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/bb488f82d9f5d160dd36834fdfbe8fb1bbce18e50473f4c17fd43264f45849c9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/43fda231dc719638180385938949bf3026f3f62cc8818e086b869b212a727d6e/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/fc0dc3eebb35af40f786fdba71d1eb20c389e435c9ec07c490ac24e7064661c3/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/46578ad330b5a98567aa441f2c2f384b21532fe7d57f895c69237accdcfd0c89/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/ab4a6835b5168db887818b30ad2b4776cfcbe25a86fde27013654616179f3b59/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/ce5e7334c803b5398676c49f04d59b1c13ee9af732dc7df1b5f0afbbb6b050a4/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/2d923ab80a88f0445d0354024034ceeb8177c21dc96b79705683b5709f13117a/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/bf1dce65016316826ba569837fa9eac35d3e81a94b926b9bcd2cf1eb09c72b6b/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/50179e9f6965ac4cfb8dda08a4f94f52ab0a45a6123709b610c967a18ce41bbc/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/cd2cb6e23f99de02f317c648ea440839f497b38c7c4590022051548b012dac40/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/33d07410a0357eb3f40876e9462b7341f1d147c0a5dc6e1e21f7df72d43a2ca4/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/fd195a5837f5d046639fa364bf3f24be248b950dda1344e396369f8603ffaa8b/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/aa9ddd639d028073b196e3ed48efb911124cf6dc9d6fd3fe944b125e0794d088/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/b18c89de1d29031b4f94ff7218a374c5ec8c04ec459dd6545bc7a37830138d0a/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/4e871c68153d29f6b1ce67e4480b928d08531ff30faea993ee43ee018c26750f/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/161e4f74da14326af500308b3084804a228b9587fa95dbff75f846e20ed230c5/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/a8fddfee0a6666d0e5dfb57dcab992f0df162731834f2d0fdca2f652bc70f46b/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/09a23a07908280b71a7cd522d6edd7dd0a985d100d9dafe5afb934aae331edea/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/a99e407e897d948fc918596722358930ffb8329d1a8806e07c5b17bbb8c1e055/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/8edb1f36117f8452ac06edd85b0b74d85448bb0f4505074ad21285e2a57daf40/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/9398ecf56cd151f8c420ec321dbfa8a8ed31a9b82f672749572634d53d3490f7/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/cc9f346401b4ad2031e6d4b26d28d2c920c4f0000a41733a43aedc0654afc87f/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/daba4fe601e0e442b18d90dd44a9d98ea207f08b09676598574531a57a4a4a21/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/7cb26c9f95d833c72766c22707c7490f580699e052c986e5cefd857d82ed0ab7/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/62cde2d55265b39faf7b9c24febff961de428349591bb324642347e2f2d2bdb5/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/dc0d0182f197a6d9919fef28ec0fcbd19c089e8898e5ead400b32aa749d559c7/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/28252caacbe2678cdc51ad737709d4d7cbae66f741bd6ccbd5b2234a1928f79c/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/4f4af5b3472ba7a341e2a60af02762f236a0103e627b187436e7dd36f1404dd0/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/8b7595c58065fc20d7dc7b9203ef53f0685a53f9470efc8306f4bc19990e00e2/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/03cfd76c15843b41fef041a5bae6a0fecbe2fd9ba6400e965c46c2814df7e2fd/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/7f1e8979d568ba86070c93765fa25122bb6a17c98cc54f3ad9bee6f84ea3d0b3/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/6c6e2c498422aa5551ecd08e34bd1325d2cb6e6fa940514106edfac52c8f9058/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/10c3ffbd2dae2c571e2f4033d6e6635e498469affa1d830d1266e567f1a0aa65/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-50:{mountpoint:/var/lib/containers/storage/overlay/085ec276857b3bc1ee20fd8604b6d22c6681547ea1b3cf4afdf9be9ecee1466e/merged major:0 minor:50 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/af567c12965aa6d7b63b8b80a24ea160ebddf2b1a67b1db8e557c3673bad7c6f/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/61556ed452cda26fd32d5004d83aef541f9c227187c6572b2dee6df4a7ac52f0/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/e89041f3f303ecee75b7fc3f83166bf298c0bf8283d0dfd3abde7005110a04b4/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/ab147ecd071cd14999ff67e2a2b57e10b1547855264702bbe6bb8a0d54d796c2/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/546d2fe8b65bd8f564d32440bae843b4c6d486edca97da05783f538d91ff717f/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/0493cc21092bfcc2bf2192686958c8997d2552db76bd75bd9b4ab37e842a7001/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/1e3b03258f4d72e1d9813d9717fac2c2c1211c43ad0988673cd922888b9b4f15/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/5e4ea86b97ad328b59245be1b23ab63b44bc4f04ebdfb6c612016a1a131d9195/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/804e323b42f86d1add4ac7736c09f10e8b98d6450209494b4d70deab6b15063d/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.119809    5033 manager.go:217] Machine: {Timestamp:2026-10-30 19:18:55.119351225 +0000 UTC m=+0.088992050 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3038891 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23bdb4743ec0a7e3b13a1886f33972 SystemUUID:ec23bdb4-743e-c0a7-e3b1-3a1886f33972 BootID:4e41448b-f5a9-42e1-849a-80b06b335c9d Filesystems:[{Device:/var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:354 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5895a27a-6712-4344-ac33-f49f30f5e920/volumes/kubernetes.io~projected/kube-api-access-l7vn7 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a98701aa-9692-47f9-be49-076a56bff175/volumes/kubernetes.io~projected/kube-api-access-9j76r DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b9c11e9813920a3cb3ca3ff7c03d3c3cee6dc391201036fdbcf502e8591ca251/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c4b76416edbd11f87689a97150a1f61d60672a3ed019fe8ba4769289735e4a2/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1dccee878e7c499b6e046ff581765482b1391356b2cdd9f3917a75637ccdcd1e/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1cccfe5457131af27e938d5d3f6d2502362c6914f287e692efa4557eb07785c/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93e5d87d97d2c039538170021c792b1fe73900d1f714ffc89a4da842fcc65b5f/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d099769215c33a05e18fc9283a4b53ec2168c17a0047ba800a25e3bd12d683ab/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cda74375-ee6c-4daa-a5dd-74d0df935567/volumes/kubernetes.io~projected/kube-api-access-4tcsb DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5/userdata/shm DeviceMajor:0 DeviceMinor:49 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/272c0c11765f9535ed78aea83df2da27573c05f7a6ec4aba6d64d906f6b1eca1/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c/volumes/kubernetes.io~projected/kube-api-access-5zst7 DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7ccee6b907b82557bb51734dfc6c64e21f928669b24cd7d3be9b76de3fb2a758/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe0bce47-ad9b-4cbe-b544-30d75ae93300/volumes/kubernetes.io~projected/kube-api-access-tvkld DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de9af921cbf267e20c250cbd0411f86cd314967771aac8f13d7f1c7f9ccd28b7/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/62223f92-bc6c-4bcd-b745-9917efe83350/volumes/kubernetes.io~projected/kube-api-access-h5bjf DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f7209f7954263dd5ab1de5181945bfc7a33d5a893df8b46c7fc4c252d333376/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f60d3c9-0f82-438d-95f9-ee13814e718b/volumes/kubernetes.io~projected/kube-api-access-vrglb DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/12011250-d65c-4fa7-9529-bd4731595e41/volumes/kubernetes.io~projected/kube-api-access-tpmd5 DeviceMajor:0 DeviceMinor:310 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52a1b4f2-94a1-4c37-b905-296e9d8e1513/volumes/kubernetes.io~projected/kube-api-access-cqt99 DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/938935ea1f15ac1f300f6c44b61b5a386d4e9588617b935bc55f6d2e7aa223a7/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52ee9927cdba9a894436437afc51d0e038a486adf23b7e0e609a197a9bba7968/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volumes/kubernetes.io~projected/kube-api-access-xpkh6 DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/438e3014-0622-4bfc-b7a0-7da9fc14472b/volumes/kubernetes.io~projected/kube-api-access-2vdrl DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-50 DeviceMajor:0 DeviceMinor:50 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/393c0959-53ac-4bb4-b156-908f228ce12e/volumes/kubernetes.io~projected/kube-api-access-lgxdh DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc012785-f079-4824-b224-044c52d1b12f/volumes/kubernetes.io~projected/kube-api-access-stvfx DeviceMajor:0 DeviceMinor:331 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89494703922da8b292acc49c688b612392c4790add457ad69d3ab8cd4d29eb71/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c520205c1a7dfd4185a5e5363a58c232c13e5ebde327c224e24ebfe5b2f26b74/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8688445669ec07307b5593d481cfef365cd6c3622416f01d24a2af47b4fe13a7/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d68860b3a523bac26c725f5c5d98e39f2f27800c44450ee547b8332bec051337/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/067b4348-3847-4370-bd9c-0995f43a8e85/volumes/kubernetes.io~projected/kube-api-access-5kpvv DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~projected/kube-api-access-lttzl DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/07071eae-8f8c-4d40-a3c3-57db59367dfb/volumes/kubernetes.io~projected/kube-api-access-hhjgv DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~projected/kube-api-access-28thn DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbec4229-cce9-465d-8ff4-c01460c9a1b9/volumes/kubernetes.io~projected/kube-api-access-f8g8f DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b509b764-b249-45ca-8c13-38332fc6c486/volumes/kubernetes.io~projected/kube-api-access-wwwjh DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3/volumes/kubernetes.io~projected/kube-api-access-sjv7l DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a93aa57250ae6c6dc0b10334f7a6f967586689ee60330a0cfd643f21604acbef/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e24911188dc9ddd41217d73c436600f3e2e6b9e7cb9405d292d647784f5c3e6e/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e968ac4ca74e41248f95ca64ee503efd8a8df657ad21a43d8d3b4321da6d2721/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f02c732c-d967-4721-acf7-76933fd1aba1/volumes/kubernetes.io~projected/kube-api-access-wj5t2 DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3fe59d-f66e-4dca-9e76-40062110cc31/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~projected/kube-api-access-cc24x DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/225f11ad869614afcbe29a191fc72f2df37697490f6a81cd0cd7abb54d2ef607/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes/kubernetes.io~projected/kube-api-access-m6ngh DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3daf3466-fda2-4bcd-9394-099194ee7c76/volumes/kubernetes.io~projected/kube-api-access-52j5x DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9b4d68d410406b19208c2b49e6902540cc6c1ca224782f3b7bd5a72d9ffe7204/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d976ba7e-0594-4893-9ef9-4332e38a27c8/volumes/kubernetes.io~projected/kube-api-access-jnmqf DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf2c3f45-829b-4784-b1da-5997daf80443/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5/volumes/kubernetes.io~projected/kube-api-access-7lsws DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0549cb11-95ca-4c56-bc03-69fe3609450d/volumes/kubernetes.io~projected/kube-api-access-tqkt7 DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dc1db0ad-000a-4196-a19f-d452c4856568/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91892ae8-06fc-4bb6-a183-c3a1c9440084/volumes/kubernetes.io~projected/kube-api-access-brbjd DeviceMajor:0 DeviceMinor:355 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4b3670eb-d511-4036-bb90-5fed8a4320b9/volumes/kubernetes.io~projected/kube-api-access-gmwzz DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/03f7a52b-5c76-43a0-8310-3c56ce2e07d2/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/63ada4981e34263626d4dbb43078aa14b1aa67b0da78f7e895da3b24bcb6519d/userdata/shm DeviceMajor:0 DeviceMinor:359 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b937576-a399-42c1-91c3-6dc38e654ce0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4c79a8c0a058853ea722f1bb9ee85744c89074802c789095906f57b2f904e17/userdata/shm DeviceMajor:0 DeviceMinor:339 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/kube-api-access-zglrx DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/72965977-109a-4e75-b0f5-31f94e98c42a/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1ad9a7a4ce9f5020c4bed6379a13e596cefa75b65701d17ed037d10375c710d7/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c46ea340-83b0-46fb-ab5e-560f43d8b4e3/volumes/kubernetes.io~projected/kube-api-access-pwdtp DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0f7209f7954263d MacAddress:d2:39:ce:21:5b:63 Speed:10000 Mtu:8901} {Name:1ad9a7a4ce9f502 MacAddress:5e:9a:0c:ab:25:33 Speed:10000 Mtu:8901} {Name:1dccee878e7c499 MacAddress:f2:a3:9f:37:16:fc Speed:10000 Mtu:8901} {Name:225f11ad869614a MacAddress:d6:3d:1b:1b:96:0e Speed:10000 Mtu:8901} {Name:3c4b76416edbd11 MacAddress:82:95:68:04:2d:21 Speed:10000 Mtu:8901} {Name:52ee9927cdba9a8 MacAddress:8e:8b:69:ce:96:6b Speed:10000 Mtu:8901} {Name:63ada4981e34263 MacAddress:52:aa:05:67:82:5d Speed:10000 Mtu:8901} {Name:8688445669ec073 MacAddress:6e:80:89:9a:08:9d Speed:10000 Mtu:8901} {Name:89494703922da8b MacAddress:4a:66:a6:b6:cc:7d Speed:10000 Mtu:8901} {Name:938935ea1f15ac1 MacAddress:4a:df:4e:63:16:76 Speed:10000 Mtu:8901} {Name:93e5d87d97d2c03 MacAddress:1a:80:9f:ce:83:8d Speed:10000 Mtu:8901} {Name:b9c11e9813920a3 MacAddress:66:3e:95:42:ab:fe Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f4:99:9a:55 Speed:0 Mtu:9001} {Name:br-int MacAddress:ea:29:68:9c:40:8d Speed:0 Mtu:8901} {Name:c520205c1a7dfd4 MacAddress:e2:0f:85:1c:7e:45 Speed:10000 Mtu:8901} {Name:d68860b3a523bac MacAddress:5e:10:da:40:b3:65 Speed:10000 Mtu:8901} {Name:e4c79a8c0a05885 MacAddress:6a:d5:4b:db:c5:9a Speed:10000 Mtu:8901} {Name:e968ac4ca74e412 MacAddress:0a:81:67:07:a5:b0 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f4:99:9a:55 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:aa:10:0f:ea:87:c1 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:1e:98:4c:94:16:52 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120138    5033 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120177    5033 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120408    5033 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120429    5033 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-135.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120553    5033 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120560    5033 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120573    5033 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120585    5033 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120602    5033 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120747    5033 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120845    5033 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120875    5033 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120888    5033 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120898    5033 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120907    5033 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:18:55.121621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.120916    5033 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:18:55.122284 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.121659    5033 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:18:55.122284 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.121687    5033 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:18:55.122284 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.122129    5033 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:18:55.124251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124237    5033 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:18:55.124251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124250    5033 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:18:55.124349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124340    5033 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124748    5033 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124793    5033 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124908    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124925    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124935    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124943    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124953    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124962    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124977    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124986    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.124997    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.125006    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.125019    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.125033    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.125064    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:18:55.125556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.125079    5033 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:18:55.128981 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.128968    5033 server.go:1263] "Started kubelet"
Oct 30 19:18:55.129074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.129029    5033 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:18:55.129136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.129067    5033 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:18:55.129189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.129136    5033 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:18:55.129435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.129415    5033 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:18:55.129471 ip-10-0-143-135 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:18:55.130133 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.130115    5033 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132124    5033 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132165    5033 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132310    5033 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:13:15 +0000 UTC" deadline="2028-05-06 15:46:18.366929715 +0000 UTC"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132339    5033 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13292h27m23.234593251s"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132367    5033 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.132374    5033 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.134018    5033 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:18:55.135145 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.134481    5033 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-135.us-east-2.compute.internal\" not found"
Oct 30 19:18:55.135630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.135382    5033 factory.go:55] Registering systemd factory
Oct 30 19:18:55.135630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.135402    5033 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:18:55.138418 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138375    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="393c0959-53ac-4bb4-b156-908f228ce12e" volumeName="kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138422    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b937576-a399-42c1-91c3-6dc38e654ce0" volumeName="kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138447    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="962ae636-bcad-4264-9b99-b47896c634e1" volumeName="kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138461    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b509b764-b249-45ca-8c13-38332fc6c486" volumeName="kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138478    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbec4229-cce9-465d-8ff4-c01460c9a1b9" volumeName="kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138492    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc012785-f079-4824-b224-044c52d1b12f" volumeName="kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx" seLinuxMountContext=""
Oct 30 19:18:55.138514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138506    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe0bce47-ad9b-4cbe-b544-30d75ae93300" volumeName="kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138524    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91892ae8-06fc-4bb6-a183-c3a1c9440084" volumeName="kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138558    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07071eae-8f8c-4d40-a3c3-57db59367dfb" volumeName="kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138577    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12011250-d65c-4fa7-9529-bd4731595e41" volumeName="kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138590    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138608    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5895a27a-6712-4344-ac33-f49f30f5e920" volumeName="kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138622    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c" volumeName="kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138636    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe0bce47-ad9b-4cbe-b544-30d75ae93300" volumeName="kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138659    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f60d3c9-0f82-438d-95f9-ee13814e718b" volumeName="kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138696    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138719    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b937576-a399-42c1-91c3-6dc38e654ce0" volumeName="kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138732    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d976ba7e-0594-4893-9ef9-4332e38a27c8" volumeName="kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138749    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc1db0ad-000a-4196-a19f-d452c4856568" volumeName="kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138763    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03f7a52b-5c76-43a0-8310-3c56ce2e07d2" volumeName="kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138776    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="438e3014-0622-4bfc-b7a0-7da9fc14472b" volumeName="kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138793    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3fe59d-f66e-4dca-9e76-40062110cc31" volumeName="kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138808    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc1db0ad-000a-4196-a19f-d452c4856568" volumeName="kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138828    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07071eae-8f8c-4d40-a3c3-57db59367dfb" volumeName="kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir" seLinuxMountContext=""
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138841    5033 factory.go:153] Registering CRI-O factory
Oct 30 19:18:55.138857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138863    5033 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138915    5033 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138932    5033 factory.go:103] Registering Raw factory
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138946    5033 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139314    5033 manager.go:319] Starting recovery of all containers
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.138843    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="438e3014-0622-4bfc-b7a0-7da9fc14472b" volumeName="kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139644    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3fe59d-f66e-4dca-9e76-40062110cc31" volumeName="kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139695    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b937576-a399-42c1-91c3-6dc38e654ce0" volumeName="kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139710    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46ea340-83b0-46fb-ab5e-560f43d8b4e3" volumeName="kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139739    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12011250-d65c-4fa7-9529-bd4731595e41" volumeName="kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139767    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139784    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3fe59d-f66e-4dca-9e76-40062110cc31" volumeName="kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 19:18:55.139808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139800    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2c3f45-829b-4784-b1da-5997daf80443" volumeName="kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139817    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0549cb11-95ca-4c56-bc03-69fe3609450d" volumeName="kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139833    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139846    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3daf3466-fda2-4bcd-9394-099194ee7c76" volumeName="kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139865    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3daf3466-fda2-4bcd-9394-099194ee7c76" volumeName="kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139877    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="438e3014-0622-4bfc-b7a0-7da9fc14472b" volumeName="kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139895    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5" volumeName="kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139909    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139922    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139940    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b3670eb-d511-4036-bb90-5fed8a4320b9" volumeName="kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139955    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72965977-109a-4e75-b0f5-31f94e98c42a" volumeName="kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139972    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3fe59d-f66e-4dca-9e76-40062110cc31" volumeName="kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.139986    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140001    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140019    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2c3f45-829b-4784-b1da-5997daf80443" volumeName="kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140032    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40f3f83-5aa4-4505-95e5-d8bc7445d5a3" volumeName="kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140051    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140064    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62223f92-bc6c-4bcd-b745-9917efe83350" volumeName="kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140078    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98701aa-9692-47f9-be49-076a56bff175" volumeName="kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140097    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f02c732c-d967-4721-acf7-76933fd1aba1" volumeName="kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140111    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0549cb11-95ca-4c56-bc03-69fe3609450d" volumeName="kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140131    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0549cb11-95ca-4c56-bc03-69fe3609450d" volumeName="kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140144    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140162    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca" seLinuxMountContext=""
Oct 30 19:18:55.140345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140183    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140240    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2c3f45-829b-4784-b1da-5997daf80443" volumeName="kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140630    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b3670eb-d511-4036-bb90-5fed8a4320b9" volumeName="kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140647    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b3670eb-d511-4036-bb90-5fed8a4320b9" volumeName="kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140666    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52a1b4f2-94a1-4c37-b905-296e9d8e1513" volumeName="kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140715    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5348f947-fe2f-4b4d-ae83-2830d5736837" volumeName="kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140735    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72965977-109a-4e75-b0f5-31f94e98c42a" volumeName="kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140749    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72965977-109a-4e75-b0f5-31f94e98c42a" volumeName="kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140763    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98701aa-9692-47f9-be49-076a56bff175" volumeName="kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140780    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46ea340-83b0-46fb-ab5e-560f43d8b4e3" volumeName="kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.140792    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca" seLinuxMountContext=""
Oct 30 19:18:55.141481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141340    5033 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141695    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98701aa-9692-47f9-be49-076a56bff175" volumeName="kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141728    5033 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141775    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141793    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f02c732c-d967-4721-acf7-76933fd1aba1" volumeName="kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141807    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe0bce47-ad9b-4cbe-b544-30d75ae93300" volumeName="kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141825    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141837    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2c3f45-829b-4784-b1da-5997daf80443" volumeName="kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141855    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="067b4348-3847-4370-bd9c-0995f43a8e85" volumeName="kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141898    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141910    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3fe59d-f66e-4dca-9e76-40062110cc31" volumeName="kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.141983    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91892ae8-06fc-4bb6-a183-c3a1c9440084" volumeName="kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.142042    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f40f3f83-5aa4-4505-95e5-d8bc7445d5a3" volumeName="kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l" seLinuxMountContext=""
Oct 30 19:18:55.142109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.142075    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0549cb11-95ca-4c56-bc03-69fe3609450d" volumeName="kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 19:18:55.144111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144085    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="172146e6-bd62-4407-93fc-19dc16979806" volumeName="kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144118    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5895a27a-6712-4344-ac33-f49f30f5e920" volumeName="kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144131    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46ea340-83b0-46fb-ab5e-560f43d8b4e3" volumeName="kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144143    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144162    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="438e3014-0622-4bfc-b7a0-7da9fc14472b" volumeName="kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144173    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91892ae8-06fc-4bb6-a183-c3a1c9440084" volumeName="kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key" seLinuxMountContext=""
Oct 30 19:18:55.144214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144184    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbec4229-cce9-465d-8ff4-c01460c9a1b9" volumeName="kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert" seLinuxMountContext=""
Oct 30 19:18:55.144502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144219    5033 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda74375-ee6c-4daa-a5dd-74d0df935567" volumeName="kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert" seLinuxMountContext=""
Oct 30 19:18:55.144502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144231    5033 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:18:55.144502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.144238    5033 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:18:55.145305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.145287    5033 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:55.148587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.148568    5033 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 19:18:55.155797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.155781    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:55.158619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.158593    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:55.159456 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.159405    5033 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:18:55.160773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.160753    5033 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:18:55.160773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.160770    5033 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:18:55.160871 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.160790    5033 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:18:55.160871 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.160831    5033 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 19:18:55.165232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.165214    5033 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:18:55.181561 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181523    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="90057d948b25a5834ecf0e48fad1e7e388d4f009aad0b752ddfd36989f4c6c2d" exitCode=0
Oct 30 19:18:55.181561 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181560    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="27cdae3d9cd6550459c4f288a3bf18c51ad27b89556153e87bc12ed63d2152c5" exitCode=0
Oct 30 19:18:55.181709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181569    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="90116250eecc250bd5e2426055f2b30d7fc3f26734a790c4369a7d1d084b9d2c" exitCode=0
Oct 30 19:18:55.181709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181575    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="f969ea909240d724b491abbe1d3ecb7fad5ef221dd508c3a5eeb64c467e4f526" exitCode=0
Oct 30 19:18:55.181709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181580    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="c1ef95c281dcc769b69652d7d6858ea3e67acad70ec208b9289d5431cfeab7f1" exitCode=0
Oct 30 19:18:55.181709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.181585    5033 generic.go:356] "Generic (PLEG): container finished" podID="0549cb11-95ca-4c56-bc03-69fe3609450d" containerID="2d8e9c4a56eee3065ce1bb0e9abaa30c0697ef48bf967b721a59ac0f56cf0125" exitCode=0
Oct 30 19:18:55.188161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.188142    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:55.188258 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.188170    5033 generic.go:356] "Generic (PLEG): container finished" podID="cf2c3f45-829b-4784-b1da-5997daf80443" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf" exitCode=255
Oct 30 19:18:55.211312 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.211259    5033 generic.go:356] "Generic (PLEG): container finished" podID="93c464cbdff5acf5d1b867b786705241" containerID="bfa6e0ad1ac35bda3f937f1a179cf1776a7bcca2b561870b56d8f20a3049baae" exitCode=0
Oct 30 19:18:55.238303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.238286    5033 manager.go:324] Recovery completed
Oct 30 19:18:55.259715 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.259690    5033 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:18:55.259715 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.259707    5033 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:18:55.259856 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.259727    5033 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:18:55.259927 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.259875    5033 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 19:18:55.260360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.259891    5033 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 19:18:55.260435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260363    5033 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 19:18:55.260435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260373    5033 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 19:18:55.260435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260381    5033 policy_none.go:50] "Start"
Oct 30 19:18:55.260435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260393    5033 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:18:55.260435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260408    5033 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:18:55.260666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260587    5033 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 19:18:55.260666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260598    5033 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 19:18:55.260666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.260606    5033 policy_none.go:44] "Start"
Oct 30 19:18:55.261847 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.261827    5033 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 19:18:55.265008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.264984    5033 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:18:55.267582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.267571    5033 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:18:55.267636 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.267593    5033 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:18:55.267636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.267601    5033 server.go:87] "Starting device plugin registration server"
Oct 30 19:18:55.267860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.267849    5033 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:18:55.267939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.267862    5033 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:18:55.268074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.268048    5033 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:18:55.268199 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.268158    5033 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:18:55.268261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.268213    5033 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:18:55.268261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.268221    5033 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:18:55.268330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.268269    5033 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:18:55.268191482Z","UUID":"678e3ed7-f0ba-47c1-94db-89d9ac5c13fd","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:18:55.269631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.269612    5033 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:18:55.269631 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.269622    5033 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:18:55.269762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.269645    5033 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:18:55.368081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.368053    5033 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:18:55.371004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.370978    5033 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:18:55.371112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.371015    5033 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:18:55.371112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.371031    5033 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-135.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:18:55.371112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.371096    5033 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.385569 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.385546    5033 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.385660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.385603    5033 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.462433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.462357    5033 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"]
Oct 30 19:18:55.463028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463013    5033 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.463188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463149    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerStarted","Data":"1a8a7f8b6b0fc1546d00e49f9dfb14cf0906f5f4b4b148156a693c6d671b4270"}
Oct 30 19:18:55.463265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463203    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerDied","Data":"bfa6e0ad1ac35bda3f937f1a179cf1776a7bcca2b561870b56d8f20a3049baae"}
Oct 30 19:18:55.463265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463235    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"93c464cbdff5acf5d1b867b786705241","Type":"ContainerStarted","Data":"972f23f78d77b7cfba04b09b5abfb86a4792cb262553aeb5c29a5a4d54a028d5"}
Oct 30 19:18:55.463265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463249    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"c51f119163141e2efcccd6dec5fb66e6","Type":"ContainerStarted","Data":"e3479eadf7b36f307a659f5a6aa94bddc967ff69239efa6d4a2ba8e6e751f88b"}
Oct 30 19:18:55.463265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463262    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal" event={"ID":"c51f119163141e2efcccd6dec5fb66e6","Type":"ContainerStarted","Data":"666586513a014414c99dfaa4e22de35127671f74d236c51b614d1c13f5383762"}
Oct 30 19:18:55.463458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.463290    5033 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.479531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.479507    5033 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:55.479613 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.479545    5033 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.479760 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.479747    5033 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:55.479812 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:55.479782    5033 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.551075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.551051    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.551155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.551091    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.551155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.551119    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652100    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652199 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652125    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652199 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652143    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652199 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652186    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652298 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652201    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93c464cbdff5acf5d1b867b786705241-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"93c464cbdff5acf5d1b867b786705241\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:55.652298 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:55.652213    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c51f119163141e2efcccd6dec5fb66e6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" (UID: \"c51f119163141e2efcccd6dec5fb66e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:56.121445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.121420    5033 apiserver.go:52] "Watching apiserver"
Oct 30 19:18:56.133181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.133157    5033 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:18:56.133860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.133832    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn","kube-system/kas-connection-checker-685895c7c9-897tx","openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb","openshift-image-registry/image-registry-5d4cb9956d-hz6xt","kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal","openshift-ingress/router-default-7d5c44fb6c-bc7gp","openshift-multus/multus-additional-cni-plugins-xc95j","openshift-console-operator/console-operator-7cd56b8dbc-b7c9l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd","openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p","openshift-network-diagnostics/network-check-target-frfms","openshift-ingress-canary/ingress-canary-pt2v2","kube-system/kas-connection-checker-685895c7c9-67jwx","openshift-ovn-kubernetes/ovnkube-node-bkpng","openshift-insights/insights-operator-7c4986d449-c8m4q","openshift-multus/multus-2tv78","openshift-dns/node-resolver-ln7fq","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal","openshift-network-operator/iptables-alerter-sdqn8","openshift-insights/periodic-gathering-v6fdz-f5wz7","kube-system/konnectivity-agent-s2mwq","openshift-image-registry/node-ca-85l8w","openshift-cluster-node-tuning-operator/tuned-tcmfn","openshift-insights/insights-runtime-extractor-b6rzj","openshift-network-console/networking-console-plugin-86c478bb88-wtwxl","kube-system/kas-connection-checker-685895c7c9-9j7fn","kube-system/global-pull-secret-syncer-5qbqv","openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb","openshift-service-ca/service-ca-5fffc97bdf-2f76w","openshift-image-registry/image-registry-648766b4bf-fs66r","openshift-dns/dns-default-nmrtr","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2","openshift-multus/network-metrics-daemon-6752x","openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"]
Oct 30 19:18:56.135868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.135846    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.136807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.136789    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.137579 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.137557    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.138378 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.138362    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.138875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.138856    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.141598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.141575    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 19:18:56.142021 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.141993    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.142145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142127    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 19:18:56.142193 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142178    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 19:18:56.142235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142212    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.142327 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142304    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 19:18:56.142398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142384    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.142445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142434    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.142521 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142314    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w7chb\""
Oct 30 19:18:56.142566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142390    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.142566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142511    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 19:18:56.142653 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142573    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.142722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142706    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.142945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142925    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.143022 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.142929    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.143137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.143121    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 19:18:56.143635 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.143613    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q64cp\""
Oct 30 19:18:56.143744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.143652    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 19:18:56.143744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.143663    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 19:18:56.143744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.143742    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7n7dp\""
Oct 30 19:18:56.144328 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144309    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 19:18:56.144732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144715    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kzmfw\""
Oct 30 19:18:56.144809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144795    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.144858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144797    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.144858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144826    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.144858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.144833    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 19:18:56.145349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145327    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 19:18:56.145505 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145485    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l242f\""
Oct 30 19:18:56.145572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145513    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 19:18:56.145572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145518    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 19:18:56.145572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145545    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.145572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145565    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.145782 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.145649    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.146180 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146162    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 19:18:56.146264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146184    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.146264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146170    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.146264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146163    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t4bhc\""
Oct 30 19:18:56.146666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146645    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.146754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146704    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 19:18:56.146754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146718    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.146754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146727    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.146754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146722    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jgwpx\""
Oct 30 19:18:56.146754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.146722    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z62xn\""
Oct 30 19:18:56.147294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.147270    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 19:18:56.147294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.147288    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.147428 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.147272    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.147428 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.147289    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.147845 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.147828    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k7ptj\""
Oct 30 19:18:56.149311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.149288    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 19:18:56.149462 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.149447    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dwcnh\""
Oct 30 19:18:56.150025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.150004    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 19:18:56.150394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.150378    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.150737 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.150720    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-sq4pq\""
Oct 30 19:18:56.150993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.150978    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.151399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.151361    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:18:56.151952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.151938    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 19:18:56.153550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.153526    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 19:18:56.153550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.153546    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.153924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.153908    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 19:18:56.154246 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154225    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 19:18:56.154340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154285    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:18:56.154340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154292    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-t757m\""
Oct 30 19:18:56.154340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154328    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt99\" (UniqueName: \"kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99\") pod \"kas-connection-checker-685895c7c9-67jwx\" (UID: \"52a1b4f2-94a1-4c37-b905-296e9d8e1513\") " pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:56.154478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154358    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxdh\" (UniqueName: \"kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh\") pod \"kas-connection-checker-685895c7c9-897tx\" (UID: \"393c0959-53ac-4bb4-b156-908f228ce12e\") " pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:56.154478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.154367    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 19:18:56.155659 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.155636    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.155967 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.155949    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:18:56.156332 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.156311    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:18:56.156480 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.156465    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t67x5\""
Oct 30 19:18:56.158111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158091    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:18:56.158111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158094    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.158345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158313    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 19:18:56.158451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158366    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:18:56.158555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158534    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.158661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158643    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.158751 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158659    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vd8dh\""
Oct 30 19:18:56.158803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158755    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:18:56.158803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158785    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:18:56.158893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158808    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158909    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158912    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159067    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158934    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158992    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159077    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vj9wr\""
Oct 30 19:18:56.159138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158656    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-n5j86\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159161    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159194    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vhzsf\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159202    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159086    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159236    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.158937    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159304    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159315    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159382    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159429    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159481    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159497    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159515    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:18:56.159513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159522    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.160016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159597    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs9gl\""
Oct 30 19:18:56.160016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159772    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 19:18:56.160016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159826    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 19:18:56.160016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.159829    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 19:18:56.161902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.161886    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxdh\" (UniqueName: \"kubernetes.io/projected/393c0959-53ac-4bb4-b156-908f228ce12e-kube-api-access-lgxdh\") pod \"kas-connection-checker-685895c7c9-897tx\" (UID: \"393c0959-53ac-4bb4-b156-908f228ce12e\") " pod="kube-system/kas-connection-checker-685895c7c9-897tx"
Oct 30 19:18:56.161902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.161897    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt99\" (UniqueName: \"kubernetes.io/projected/52a1b4f2-94a1-4c37-b905-296e9d8e1513-kube-api-access-cqt99\") pod \"kas-connection-checker-685895c7c9-67jwx\" (UID: \"52a1b4f2-94a1-4c37-b905-296e9d8e1513\") " pod="kube-system/kas-connection-checker-685895c7c9-67jwx"
Oct 30 19:18:56.170169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.170153    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pt2v2_cbec4229-cce9-465d-8ff4-c01460c9a1b9/serve-healthcheck-canary/0.log"
Oct 30 19:18:56.229550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.229529    5033 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:56.229711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.229694    5033 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:56.236323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.236307    5033 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:18:56.243744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.243726    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:18:56.244072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244050    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktbkj\""
Oct 30 19:18:56.244072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244058    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:18:56.244223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244074    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:18:56.244223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244095    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 19:18:56.244223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244096    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:18:56.244459 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244442    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctbj8\""
Oct 30 19:18:56.244459 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244446    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zxgfv\""
Oct 30 19:18:56.244562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244473    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bjlqm\""
Oct 30 19:18:56.244562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244526    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mgmzn\""
Oct 30 19:18:56.244644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244572    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkcxf\""
Oct 30 19:18:56.244644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244579    5033 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:56.244644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244585    5033 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:18:56.244644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244596    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rhqgv\""
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244667    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dk8pw\""
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.244714    5033 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244729    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244767    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4vgss\""
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.244812    5033 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal"
Oct 30 19:18:56.244843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.244843    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6rfl7\""
Oct 30 19:18:56.254978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.254960    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc24x\" (UniqueName: \"kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.255042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.254990    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.255042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255011    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.255042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255026    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.255162 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255049    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.255230 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255213    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.255268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255227    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-config\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.255268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255244    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.255339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255292    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.255339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255326    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.255435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255418    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.255435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255428    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-env-overrides\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.255501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255435    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.255501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255472    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.255501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255495    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.255602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255511    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjgv\" (UniqueName: \"kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.255602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255527    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28thn\" (UniqueName: \"kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.255602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255542    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.255602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255557    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.255602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255579    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255630    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255637    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-tuned\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255710    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255731    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255750    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255789    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpvv\" (UniqueName: \"kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.255840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255827    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255861    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255890    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255925    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5bjf\" (UniqueName: \"kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf\") pod \"network-check-source-6f78d59f99-vkgxb\" (UID: \"62223f92-bc6c-4bcd-b745-9917efe83350\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255931    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-default-certificate\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255950    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zst7\" (UniqueName: \"kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255975    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.255998    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256022    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256055    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256063    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-service-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256079    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.256104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256080    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256131    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256215    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256232    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-daemon-config\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256248    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256280    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256301    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cni-binary-copy\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256303    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256405    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256432    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256460    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256474    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256491    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256519    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256553    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttzl\" (UniqueName: \"kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256558    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3daf3466-fda2-4bcd-9394-099194ee7c76-iptables-alerter-script\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256579    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmwzz\" (UniqueName: \"kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256628    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0bce47-ad9b-4cbe-b544-30d75ae93300-tmp-dir\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.256660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256658    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256731    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256759    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256782    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8g8f\" (UniqueName: \"kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256807    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stvfx\" (UniqueName: \"kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx\") pod \"migrator-66df5d496b-sd6pb\" (UID: \"cc012785-f079-4824-b224-044c52d1b12f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256835    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256860    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256884    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256908    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda74375-ee6c-4daa-a5dd-74d0df935567-serving-cert\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256945    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256974    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.256990    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b937576-a399-42c1-91c3-6dc38e654ce0-tmp\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257013    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpkh6\" (UniqueName: \"kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257026    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf2c3f45-829b-4784-b1da-5997daf80443-trusted-ca\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257039    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257059    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-tmp\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257065    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257087    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257103    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257120    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257141    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257176    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257206    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257242    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257289    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbec4229-cce9-465d-8ff4-c01460c9a1b9-cert\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257295    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257322    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/962ae636-bcad-4264-9b99-b47896c634e1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257348    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257357    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-original-pull-secret\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257420    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257469    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.257477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257478    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257513    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0bce47-ad9b-4cbe-b544-30d75ae93300-config-volume\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257532    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257575    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257722    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257735    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwwjh\" (UniqueName: \"kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh\") pod \"volume-data-source-validator-58c9f5f46d-g46gd\" (UID: \"b509b764-b249-45ca-8c13-38332fc6c486\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257756    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257772    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257790    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257817    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257842    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257859    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257868    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257876    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257899    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257930    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257961    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257969    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.257972    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258051    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258086    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b3670eb-d511-4036-bb90-5fed8a4320b9-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258092    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmd5\" (UniqueName: \"kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258213    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258271    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258276    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258313    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258338    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258367    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258393    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258447    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258543    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0549cb11-95ca-4c56-bc03-69fe3609450d-cni-binary-copy\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258551    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258564    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258566    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258599    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258618    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258662    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a98701aa-9692-47f9-be49-076a56bff175-serving-cert\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258660    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258736    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258768    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkt7\" (UniqueName: \"kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258795    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258839    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258864    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258863    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258891    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvkld\" (UniqueName: \"kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.258920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258927    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.258999    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrglb\" (UniqueName: \"kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb\") pod \"migrator-66df5d496b-k2nfn\" (UID: \"3f60d3c9-0f82-438d-95f9-ee13814e718b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259024    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259044    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259060    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259075    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjv7l\" (UniqueName: \"kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259107    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52j5x\" (UniqueName: \"kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259194    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f02c732c-d967-4721-acf7-76933fd1aba1-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259231    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259267    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259287    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-config\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259292    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnmqf\" (UniqueName: \"kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf\") pod \"kas-connection-checker-685895c7c9-9j7fn\" (UID: \"d976ba7e-0594-4893-9ef9-4332e38a27c8\") " pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259321    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbjd\" (UniqueName: \"kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259372    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259405    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259433    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259457    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259464    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc1db0ad-000a-4196-a19f-d452c4856568-konnectivity-ca\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259474    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc1db0ad-000a-4196-a19f-d452c4856568-agent-certs\") pod \"konnectivity-agent-s2mwq\" (UID: \"dc1db0ad-000a-4196-a19f-d452c4856568\") " pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259510    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259542    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259574    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259615    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259665    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259714    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259725    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-stats-auth\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259742    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259767    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259768    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cda74375-ee6c-4daa-a5dd-74d0df935567-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259790    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259851    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259853    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5895a27a-6712-4344-ac33-f49f30f5e920-serviceca\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259853    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b3670eb-d511-4036-bb90-5fed8a4320b9-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259891    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf2c3f45-829b-4784-b1da-5997daf80443-serving-cert\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.259950    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260014    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260052    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260086    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7vn7\" (UniqueName: \"kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260094    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a98701aa-9692-47f9-be49-076a56bff175-config\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260120    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260157    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovnkube-script-lib\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260228    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260249    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260276    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260306    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260344    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260373    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260487    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260533    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12011250-d65c-4fa7-9529-bd4731595e41-data-volume\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260583    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcsb\" (UniqueName: \"kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260602    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260653    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b3fe59d-f66e-4dca-9e76-40062110cc31-ovn-node-metrics-cert\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260691    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-cabundle\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260707    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.260704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260728    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260748    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260783    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5t2\" (UniqueName: \"kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260804    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260828    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260871    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260892    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91892ae8-06fc-4bb6-a183-c3a1c9440084-signing-key\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260950    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdtp\" (UniqueName: \"kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.260972    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cda74375-ee6c-4daa-a5dd-74d0df935567-snapshots\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261001    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261129    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261175    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j76r\" (UniqueName: \"kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261232    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261263    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261316    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07071eae-8f8c-4d40-a3c3-57db59367dfb-tmp-dir\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.262493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.261372    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.263473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.263452    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjgv\" (UniqueName: \"kubernetes.io/projected/07071eae-8f8c-4d40-a3c3-57db59367dfb-kube-api-access-hhjgv\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.263539 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.263523    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc24x\" (UniqueName: \"kubernetes.io/projected/cf2c3f45-829b-4784-b1da-5997daf80443-kube-api-access-cc24x\") pod \"console-operator-7cd56b8dbc-b7c9l\" (UID: \"cf2c3f45-829b-4784-b1da-5997daf80443\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.264316 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.264297    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28thn\" (UniqueName: \"kubernetes.io/projected/72965977-109a-4e75-b0f5-31f94e98c42a-kube-api-access-28thn\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.264547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.264529    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.264713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.264696    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmwzz\" (UniqueName: \"kubernetes.io/projected/4b3670eb-d511-4036-bb90-5fed8a4320b9-kube-api-access-gmwzz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgws2\" (UID: \"4b3670eb-d511-4036-bb90-5fed8a4320b9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2"
Oct 30 19:18:56.264824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.264806    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttzl\" (UniqueName: \"kubernetes.io/projected/8b937576-a399-42c1-91c3-6dc38e654ce0-kube-api-access-lttzl\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.264869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.264806    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpvv\" (UniqueName: \"kubernetes.io/projected/067b4348-3847-4370-bd9c-0995f43a8e85-kube-api-access-5kpvv\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.265122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.265106    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5bjf\" (UniqueName: \"kubernetes.io/projected/62223f92-bc6c-4bcd-b745-9917efe83350-kube-api-access-h5bjf\") pod \"network-check-source-6f78d59f99-vkgxb\" (UID: \"62223f92-bc6c-4bcd-b745-9917efe83350\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vkgxb"
Oct 30 19:18:56.266943 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266798    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwwjh\" (UniqueName: \"kubernetes.io/projected/b509b764-b249-45ca-8c13-38332fc6c486-kube-api-access-wwwjh\") pod \"volume-data-source-validator-58c9f5f46d-g46gd\" (UID: \"b509b764-b249-45ca-8c13-38332fc6c486\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g46gd"
Oct 30 19:18:56.266943 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266830    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmd5\" (UniqueName: \"kubernetes.io/projected/12011250-d65c-4fa7-9529-bd4731595e41-kube-api-access-tpmd5\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.266943 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266914    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpkh6\" (UniqueName: \"kubernetes.io/projected/7b3fe59d-f66e-4dca-9e76-40062110cc31-kube-api-access-xpkh6\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.267091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266916    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zst7\" (UniqueName: \"kubernetes.io/projected/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-kube-api-access-5zst7\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.267091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266971    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stvfx\" (UniqueName: \"kubernetes.io/projected/cc012785-f079-4824-b224-044c52d1b12f-kube-api-access-stvfx\") pod \"migrator-66df5d496b-sd6pb\" (UID: \"cc012785-f079-4824-b224-044c52d1b12f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sd6pb"
Oct 30 19:18:56.267091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.266979    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.267091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.267048    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"periodic-gathering-v6fdz-f5wz7\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") " pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:18:56.267231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.267182    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8g8f\" (UniqueName: \"kubernetes.io/projected/cbec4229-cce9-465d-8ff4-c01460c9a1b9-kube-api-access-f8g8f\") pod \"ingress-canary-pt2v2\" (UID: \"cbec4229-cce9-465d-8ff4-c01460c9a1b9\") " pod="openshift-ingress-canary/ingress-canary-pt2v2"
Oct 30 19:18:56.267268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.267247    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkt7\" (UniqueName: \"kubernetes.io/projected/0549cb11-95ca-4c56-bc03-69fe3609450d-kube-api-access-tqkt7\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.267533 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.267517    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvkld\" (UniqueName: \"kubernetes.io/projected/fe0bce47-ad9b-4cbe-b544-30d75ae93300-kube-api-access-tvkld\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.267869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.267851    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnmqf\" (UniqueName: \"kubernetes.io/projected/d976ba7e-0594-4893-9ef9-4332e38a27c8-kube-api-access-jnmqf\") pod \"kas-connection-checker-685895c7c9-9j7fn\" (UID: \"d976ba7e-0594-4893-9ef9-4332e38a27c8\") " pod="kube-system/kas-connection-checker-685895c7c9-9j7fn"
Oct 30 19:18:56.268553 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.268536    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.269729    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdtp\" (UniqueName: \"kubernetes.io/projected/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-kube-api-access-pwdtp\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.269818    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52j5x\" (UniqueName: \"kubernetes.io/projected/3daf3466-fda2-4bcd-9394-099194ee7c76-kube-api-access-52j5x\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.269819    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrglb\" (UniqueName: \"kubernetes.io/projected/3f60d3c9-0f82-438d-95f9-ee13814e718b-kube-api-access-vrglb\") pod \"migrator-66df5d496b-k2nfn\" (UID: \"3f60d3c9-0f82-438d-95f9-ee13814e718b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k2nfn"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270081    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270569    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsws\" (UniqueName: \"kubernetes.io/projected/46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5-kube-api-access-7lsws\") pod \"network-check-target-frfms\" (UID: \"46297d6c-6db8-4e9d-9e74-8bc1a53fa7e5\") " pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270646    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcsb\" (UniqueName: \"kubernetes.io/projected/cda74375-ee6c-4daa-a5dd-74d0df935567-kube-api-access-4tcsb\") pod \"insights-operator-7c4986d449-c8m4q\" (UID: \"cda74375-ee6c-4daa-a5dd-74d0df935567\") " pod="openshift-insights/insights-operator-7c4986d449-c8m4q"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270654    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbjd\" (UniqueName: \"kubernetes.io/projected/91892ae8-06fc-4bb6-a183-c3a1c9440084-kube-api-access-brbjd\") pod \"service-ca-5fffc97bdf-2f76w\" (UID: \"91892ae8-06fc-4bb6-a183-c3a1c9440084\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2f76w"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270767    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5t2\" (UniqueName: \"kubernetes.io/projected/f02c732c-d967-4721-acf7-76933fd1aba1-kube-api-access-wj5t2\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.270901    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjv7l\" (UniqueName: \"kubernetes.io/projected/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-kube-api-access-sjv7l\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.271232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.271126    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7vn7\" (UniqueName: \"kubernetes.io/projected/5895a27a-6712-4344-ac33-f49f30f5e920-kube-api-access-l7vn7\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.272941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.271828    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j76r\" (UniqueName: \"kubernetes.io/projected/a98701aa-9692-47f9-be49-076a56bff175-kube-api-access-9j76r\") pod \"service-ca-operator-544d749c8c-4d74p\" (UID: \"a98701aa-9692-47f9-be49-076a56bff175\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p"
Oct 30 19:18:56.362061 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362041    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.362061 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362066    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362081    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362095    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362113    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362136    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362163    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-run\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362174    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-kubelet\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362166    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-log-socket\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362174    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.362209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362195    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-ovn\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362201    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362226    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362244    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362251    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.362258    5033 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362273    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362281    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-registration-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362298    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.362329    5033 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362341    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-node-log\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362341    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-host\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.362333    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:56.862312724 +0000 UTC m=+1.831953560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362392    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.362432    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:56.862418038 +0000 UTC m=+1.832058854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : secret "metrics-daemon-secret" not found
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362508    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362542    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362566    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.362576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362579    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3daf3466-fda2-4bcd-9394-099194ee7c76-host-slash\") pod \"iptables-alerter-sdqn8\" (UID: \"3daf3466-fda2-4bcd-9394-099194ee7c76\") " pod="openshift-network-operator/iptables-alerter-sdqn8"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362590    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362596    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-hostroot\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362610    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5895a27a-6712-4344-ac33-f49f30f5e920-host\") pod \"node-ca-85l8w\" (UID: \"5895a27a-6712-4344-ac33-f49f30f5e920\") " pod="openshift-image-registry/node-ca-85l8w"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362615    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362627    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-k8s-cni-cncf-io\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362637    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-etc-kubernetes\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362638    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362689    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-slash\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362696    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362716    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362731    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362744    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362747    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-systemd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362764    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362787    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362790    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-var-lib-kubelet\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362798    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-sys\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362812    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362834    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362839    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-netd\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362846    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12011250-d65c-4fa7-9529-bd4731595e41-crio-socket\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362856    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362884    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-modprobe-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362920    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-cnibin\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362922    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362931    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12011250-d65c-4fa7-9529-bd4731595e41-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362954    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-etc-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362959    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysconfig\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362965    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.362994    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363005    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363035    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363061    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.363077    5033 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363069    5033 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363088    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.363114    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:56.863100702 +0000 UTC m=+1.832741529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363126    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363133    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-multus\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363144    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363159    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363181    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-kubelet\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363181    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363201    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-conf\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363212    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-ovn-kubernetes\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363215    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363217    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07071eae-8f8c-4d40-a3c3-57db59367dfb-hosts-file\") pod \"node-resolver-ln7fq\" (UID: \"07071eae-8f8c-4d40-a3c3-57db59367dfb\") " pod="openshift-dns/node-resolver-ln7fq"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363240    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363252    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-etc-selinux\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363258    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-socket-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363269    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-var-lib-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363277    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363304    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363320    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363333    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363356    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363378    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-conf-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363384    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363423    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-device-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363427    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363359    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-kubernetes\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363428    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-multus-socket-dir-parent\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363386    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-systemd\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363458    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-run-netns\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363459    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363492    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363507    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.363521    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:56.863512159 +0000 UTC m=+1.833153001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363547    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.363501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363546    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-dbus\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363589    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03f7a52b-5c76-43a0-8310-3c56ce2e07d2-kubelet-config\") pod \"global-pull-secret-syncer-5qbqv\" (UID: \"03f7a52b-5c76-43a0-8310-3c56ce2e07d2\") " pod="kube-system/global-pull-secret-syncer-5qbqv"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363582    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-systemd-units\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363623    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363643    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363685    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-run-openvswitch\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363667    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363728    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363750    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363773    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363784    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-lib-modules\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363799    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363820    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363832    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-sys-fs\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363845    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363870    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363879    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-cnibin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363888    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0549cb11-95ca-4c56-bc03-69fe3609450d-system-cni-dir\") pod \"multus-additional-cni-plugins-xc95j\" (UID: \"0549cb11-95ca-4c56-bc03-69fe3609450d\") " pod="openshift-multus/multus-additional-cni-plugins-xc95j"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363895    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363916    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b3fe59d-f66e-4dca-9e76-40062110cc31-host-cni-bin\") pod \"ovnkube-node-bkpng\" (UID: \"7b3fe59d-f66e-4dca-9e76-40062110cc31\") " pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363920    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b937576-a399-42c1-91c3-6dc38e654ce0-etc-sysctl-d\") pod \"tuned-tcmfn\" (UID: \"8b937576-a399-42c1-91c3-6dc38e654ce0\") " pod="openshift-cluster-node-tuning-operator/tuned-tcmfn"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363936    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/067b4348-3847-4370-bd9c-0995f43a8e85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s25lf\" (UID: \"067b4348-3847-4370-bd9c-0995f43a8e85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s25lf"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363936    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.363977    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364003    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364019    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-system-cni-dir\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364027    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.363981    5033 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.364073    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:56.864059505 +0000 UTC m=+1.833700339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364071    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-var-lib-cni-bin\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364093    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-multus-certs\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.364258    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c46ea340-83b0-46fb-ab5e-560f43d8b4e3-host-run-netns\") pod \"multus-2tv78\" (UID: \"c46ea340-83b0-46fb-ab5e-560f43d8b4e3\") " pod="openshift-multus/multus-2tv78"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.365821    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f02c732c-d967-4721-acf7-76933fd1aba1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-g24tb\" (UID: \"f02c732c-d967-4721-acf7-76933fd1aba1\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.365843    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"image-registry-648766b4bf-fs66r\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") " pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.366240    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f40f3f83-5aa4-4505-95e5-d8bc7445d5a3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2mnzx\" (UID: \"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.366383    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"image-registry-5d4cb9956d-hz6xt\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") " pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.366473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.366476    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe0bce47-ad9b-4cbe-b544-30d75ae93300-metrics-tls\") pod \"dns-default-nmrtr\" (UID: \"fe0bce47-ad9b-4cbe-b544-30d75ae93300\") " pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.371102 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.371083    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-k2nfn_3f60d3c9-0f82-438d-95f9-ee13814e718b/migrator/0.log"
Oct 30 19:18:56.437021 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.436968    5033 scope.go:122] "RemoveContainer" containerID="23f10120be50997a4855f1a481bcb0608fc9186065e68eef8b2bb0a0c0a046cf"
Oct 30 19:18:56.437119 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.437027    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"
Oct 30 19:18:56.437825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.437807    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"
Oct 30 19:18:56.438942 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.438928    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:18:56.447020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.447004    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:56.451143 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.451129    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:18:56.571901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.571877    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-sd6pb_cc012785-f079-4824-b224-044c52d1b12f/migrator/0.log"
Oct 30 19:18:56.736005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.735977    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb"]
Oct 30 19:18:56.739335 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:56.739305    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02c732c_d967_4721_acf7_76933fd1aba1.slice/crio-35571d57ea4f70eabc5522a1cbcb9185535c7657a6a9ebdc464c031d7ae3aaf3 WatchSource:0}: Error finding container 35571d57ea4f70eabc5522a1cbcb9185535c7657a6a9ebdc464c031d7ae3aaf3: Status 404 returned error can't find the container with id 35571d57ea4f70eabc5522a1cbcb9185535c7657a6a9ebdc464c031d7ae3aaf3
Oct 30 19:18:56.741441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.741422    5033 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:18:56.746412 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.746394    5033 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:18:56.769960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.769939    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx"]
Oct 30 19:18:56.777960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.777931    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-fgws2_4b3670eb-d511-4036-bb90-5fed8a4320b9/kube-storage-version-migrator-operator/0.log"
Oct 30 19:18:56.821497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.821474    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nmrtr"]
Oct 30 19:18:56.823958 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:56.823935    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0bce47_ad9b_4cbe_b544_30d75ae93300.slice/crio-10cbbaffc43983285d70df89fa2865a8136347f694cc46050a1202107cdbaff4 WatchSource:0}: Error finding container 10cbbaffc43983285d70df89fa2865a8136347f694cc46050a1202107cdbaff4: Status 404 returned error can't find the container with id 10cbbaffc43983285d70df89fa2865a8136347f694cc46050a1202107cdbaff4
Oct 30 19:18:56.827419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.827396    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:18:56.830174 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:56.830136    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5348f947_fe2f_4b4d_ae83_2830d5736837.slice/crio-6f82fd3cb76ea40fb110f44d2816a01d05c3121e432593c301c3d318143d3276 WatchSource:0}: Error finding container 6f82fd3cb76ea40fb110f44d2816a01d05c3121e432593c301c3d318143d3276: Status 404 returned error can't find the container with id 6f82fd3cb76ea40fb110f44d2816a01d05c3121e432593c301c3d318143d3276
Oct 30 19:18:56.860160 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.860143    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:18:56.865728 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:18:56.865703    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172146e6_bd62_4407_93fc_19dc16979806.slice/crio-76561b593be4efb295fc15ebb3f2292691d6482c26cf0e6d5b89b8b18d3dc065 WatchSource:0}: Error finding container 76561b593be4efb295fc15ebb3f2292691d6482c26cf0e6d5b89b8b18d3dc065: Status 404 returned error can't find the container with id 76561b593be4efb295fc15ebb3f2292691d6482c26cf0e6d5b89b8b18d3dc065
Oct 30 19:18:56.867853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.867835    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.867914 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.867864    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:56.867914 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.867907    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:56.867984 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.867959    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:56.867984 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.867971    5033 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:56.868073 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868024    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.868006268 +0000 UTC m=+2.837647099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:56.868073 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868051    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.86804275 +0000 UTC m=+2.837683564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : configmap references non-existent config key: service-ca.crt
Oct 30 19:18:56.868073 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868056    5033 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 19:18:56.868073 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868066    5033 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:56.868241 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868113    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.868097046 +0000 UTC m=+2.837737866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : secret "metrics-daemon-secret" not found
Oct 30 19:18:56.868241 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868132    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.868122361 +0000 UTC m=+2.837763180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:56.868241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.868153    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:56.868334 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868258    5033 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:56.868334 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:56.868294    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:57.86828365 +0000 UTC m=+2.837924465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:56.925858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:56.925835    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:18:57.236107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.236051    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" event={"ID":"5348f947-fe2f-4b4d-ae83-2830d5736837","Type":"ContainerStarted","Data":"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"}
Oct 30 19:18:57.236576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.236119    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" event={"ID":"5348f947-fe2f-4b4d-ae83-2830d5736837","Type":"ContainerStarted","Data":"6f82fd3cb76ea40fb110f44d2816a01d05c3121e432593c301c3d318143d3276"}
Oct 30 19:18:57.238154 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.238134    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:18:57.238265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.238243    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l" event={"ID":"cf2c3f45-829b-4784-b1da-5997daf80443","Type":"ContainerStarted","Data":"93f0c674978534222aed607b37cffb95b6a56c6c32e65e471818ec7cda73b1f4"}
Oct 30 19:18:57.239468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.239417    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx" event={"ID":"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3","Type":"ContainerStarted","Data":"7dbfab1c74c7c181f9cbbd843b80883a079044f205a5af7d9b09bbefcaa5a2e9"}
Oct 30 19:18:57.241351 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.241332    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" event={"ID":"172146e6-bd62-4407-93fc-19dc16979806","Type":"ContainerStarted","Data":"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"}
Oct 30 19:18:57.241469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.241363    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" event={"ID":"172146e6-bd62-4407-93fc-19dc16979806","Type":"ContainerStarted","Data":"76561b593be4efb295fc15ebb3f2292691d6482c26cf0e6d5b89b8b18d3dc065"}
Oct 30 19:18:57.242548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.242528    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmrtr" event={"ID":"fe0bce47-ad9b-4cbe-b544-30d75ae93300","Type":"ContainerStarted","Data":"10cbbaffc43983285d70df89fa2865a8136347f694cc46050a1202107cdbaff4"}
Oct 30 19:18:57.243749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.243721    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb" event={"ID":"f02c732c-d967-4721-acf7-76933fd1aba1","Type":"ContainerStarted","Data":"35571d57ea4f70eabc5522a1cbcb9185535c7657a6a9ebdc464c031d7ae3aaf3"}
Oct 30 19:18:57.890750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.890711    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.890794    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.890845    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.890857    5033 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.890885    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.890909    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:57.890935 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.890924    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.890906418 +0000 UTC m=+4.860547252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.890969    5033 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.891002    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs podName:5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.890992017 +0000 UTC m=+4.860632844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs") pod "network-metrics-daemon-6752x" (UID: "5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c") : secret "metrics-daemon-secret" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.891028    5033 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.891049    5033 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.891080    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.891070253 +0000 UTC m=+4.860711081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:57.891244 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:57.891095    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:18:59.89108665 +0000 UTC m=+4.860727466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:57.891634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:57.891611    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72965977-109a-4e75-b0f5-31f94e98c42a-service-ca-bundle\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:58.251157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:58.251129    5033 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:58.251627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:58.251129    5033 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:58.251709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:58.251141    5033 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:18:59.462329 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.462303    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.913192    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.913260    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.913339    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913366    5033 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.913378    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913442    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert podName:962ae636-bcad-4264-9b99-b47896c634e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:03.913417287 +0000 UTC m=+8.883058122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-wtwxl" (UID: "962ae636-bcad-4264-9b99-b47896c634e1") : secret "networking-console-plugin-cert" not found
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913455    5033 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913469    5033 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913502    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls podName:12011250-d65c-4fa7-9529-bd4731595e41 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:03.913490429 +0000 UTC m=+8.883131262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6rzj" (UID: "12011250-d65c-4fa7-9529-bd4731595e41") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:18:59.913524 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:18:59.913519    5033 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs podName:72965977-109a-4e75-b0f5-31f94e98c42a nodeName:}" failed. No retries permitted until 2026-10-30 19:19:03.913510383 +0000 UTC m=+8.883151196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs") pod "router-default-7d5c44fb6c-bc7gp" (UID: "72965977-109a-4e75-b0f5-31f94e98c42a") : secret "router-metrics-certs-default" not found
Oct 30 19:18:59.915913 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:18:59.915863    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c-metrics-certs\") pod \"network-metrics-daemon-6752x\" (UID: \"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c\") " pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:19:00.046333 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.046254    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6752x"
Oct 30 19:19:00.263154 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.263110    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmrtr" event={"ID":"fe0bce47-ad9b-4cbe-b544-30d75ae93300","Type":"ContainerStarted","Data":"7098ea6e0582978063525401adad624f41b208129b881f8e3fa56a18a5d41c29"}
Oct 30 19:19:00.263313 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.263175    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmrtr" event={"ID":"fe0bce47-ad9b-4cbe-b544-30d75ae93300","Type":"ContainerStarted","Data":"6ed428f303b7e6a10d9772aea833f088a55a8da7718c6c7f8440b32e06e7e21b"}
Oct 30 19:19:00.263313 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.263229    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:19:00.264720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.264696    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-g24tb" event={"ID":"f02c732c-d967-4721-acf7-76933fd1aba1","Type":"ContainerStarted","Data":"76e6defca25e9bdb3a5c459a815986fe4e6b0958aca4141a3759d5e1138138cd"}
Oct 30 19:19:00.266277 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.266218    5033 generic.go:356] "Generic (PLEG): container finished" podID="a98701aa-9692-47f9-be49-076a56bff175" containerID="d06ecc7c78f584180cb8cbfc443aba8afe39b4cba6b7b2ac2ae2f46a42e965ad" exitCode=0
Oct 30 19:19:00.266400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.266275    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p" event={"ID":"a98701aa-9692-47f9-be49-076a56bff175","Type":"ContainerDied","Data":"d06ecc7c78f584180cb8cbfc443aba8afe39b4cba6b7b2ac2ae2f46a42e965ad"}
Oct 30 19:19:00.266633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.266615    5033 scope.go:122] "RemoveContainer" containerID="d06ecc7c78f584180cb8cbfc443aba8afe39b4cba6b7b2ac2ae2f46a42e965ad"
Oct 30 19:19:00.270869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.270845    5033 generic.go:356] "Generic (PLEG): container finished" podID="cda74375-ee6c-4daa-a5dd-74d0df935567" containerID="6d3aa2c33b1965aef8c039e2c9276c124fda472d60ac8955883bea3976a2108b" exitCode=0
Oct 30 19:19:00.270989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.270870    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-c8m4q" event={"ID":"cda74375-ee6c-4daa-a5dd-74d0df935567","Type":"ContainerDied","Data":"6d3aa2c33b1965aef8c039e2c9276c124fda472d60ac8955883bea3976a2108b"}
Oct 30 19:19:00.271556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.271243    5033 scope.go:122] "RemoveContainer" containerID="6d3aa2c33b1965aef8c039e2c9276c124fda472d60ac8955883bea3976a2108b"
Oct 30 19:19:00.274343 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.274316    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx" event={"ID":"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3","Type":"ContainerStarted","Data":"90f4111b0922e2ac02f1af64e8927524cd9aa0e69889e0054a0e7b4ea0289f9b"}
Oct 30 19:19:00.274441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.274368    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2mnzx" event={"ID":"f40f3f83-5aa4-4505-95e5-d8bc7445d5a3","Type":"ContainerStarted","Data":"ac0d2a819d79890b76a7efa536c3366883d88a7a1ba8efd1edd17037e4fd4af3"}
Oct 30 19:19:00.342798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.342772    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6752x"]
Oct 30 19:19:00.555105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:00.555071    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:19:01.280331 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.280114    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-4d74p" event={"ID":"a98701aa-9692-47f9-be49-076a56bff175","Type":"ContainerStarted","Data":"337d244eec17e20519573263777765a2657e0bab5f3b50349f7207ad3836a24e"}
Oct 30 19:19:01.282058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.282036    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-c8m4q" event={"ID":"cda74375-ee6c-4daa-a5dd-74d0df935567","Type":"ContainerStarted","Data":"1aafd2ef4c7b49fb2edf93ee9d84ce20f15b1dc9733bbd9cedd712b80fb40482"}
Oct 30 19:19:01.283288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.283262    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6752x" event={"ID":"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c","Type":"ContainerStarted","Data":"493bcec428db4595e91f94988665c5cdf05231bc7d1facb51e603891252529e0"}
Oct 30 19:19:01.388933 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.388897    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:19:01.389186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.389169    5033 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:19:01.723116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:01.723085    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:19:02.288172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:02.288132    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6752x" event={"ID":"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c","Type":"ContainerStarted","Data":"69ba807a2a27001ae0c0626b7a984e2635232687f74c3d35dcd0c44fff72b3d2"}
Oct 30 19:19:02.288327 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:02.288193    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6752x" event={"ID":"5fac3b5d-8970-48d1-8bde-bcdad3bd1e4c","Type":"ContainerStarted","Data":"9c632bad53762e503dfc84f30408b6edbd217f20ea9a2e59ec00981d99a3ecf0"}
Oct 30 19:19:03.946406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.946370    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:19:03.947012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.946415    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:03.947012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.946443    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:19:03.948837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.948803    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12011250-d65c-4fa7-9529-bd4731595e41-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6rzj\" (UID: \"12011250-d65c-4fa7-9529-bd4731595e41\") " pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:19:03.948969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.948932    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72965977-109a-4e75-b0f5-31f94e98c42a-metrics-certs\") pod \"router-default-7d5c44fb6c-bc7gp\" (UID: \"72965977-109a-4e75-b0f5-31f94e98c42a\") " pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:03.948969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.948958    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/962ae636-bcad-4264-9b99-b47896c634e1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-wtwxl\" (UID: \"962ae636-bcad-4264-9b99-b47896c634e1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:19:03.951743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:03.951725    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6rzj"
Oct 30 19:19:04.227147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.227123    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6rzj"]
Oct 30 19:19:04.229488 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:19:04.229464    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12011250_d65c_4fa7_9529_bd4731595e41.slice/crio-5c24c3a718e438ae27fd7d55b35e481bf8744748c3faddea7825a72ba52a7ca9 WatchSource:0}: Error finding container 5c24c3a718e438ae27fd7d55b35e481bf8744748c3faddea7825a72ba52a7ca9: Status 404 returned error can't find the container with id 5c24c3a718e438ae27fd7d55b35e481bf8744748c3faddea7825a72ba52a7ca9
Oct 30 19:19:04.236804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.236783    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:04.239879 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.239809    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"
Oct 30 19:19:04.305928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.303481    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6rzj" event={"ID":"12011250-d65c-4fa7-9529-bd4731595e41","Type":"ContainerStarted","Data":"5c24c3a718e438ae27fd7d55b35e481bf8744748c3faddea7825a72ba52a7ca9"}
Oct 30 19:19:04.526099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.526025    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-wtwxl"]
Oct 30 19:19:04.529160 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:19:04.529129    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962ae636_bcad_4264_9b99_b47896c634e1.slice/crio-f26f77fb3d6512a2ff71dc7aed12994befe6c979e70f7b0430c581dd6e4393a1 WatchSource:0}: Error finding container f26f77fb3d6512a2ff71dc7aed12994befe6c979e70f7b0430c581dd6e4393a1: Status 404 returned error can't find the container with id f26f77fb3d6512a2ff71dc7aed12994befe6c979e70f7b0430c581dd6e4393a1
Oct 30 19:19:04.529960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.529934    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d5c44fb6c-bc7gp"]
Oct 30 19:19:04.533805 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:19:04.533783    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72965977_109a_4e75_b0f5_31f94e98c42a.slice/crio-101b0c845d2a73ee49a8c2ace70a9c3d12c48e5330f2dd89221109b2ef7b6e7d WatchSource:0}: Error finding container 101b0c845d2a73ee49a8c2ace70a9c3d12c48e5330f2dd89221109b2ef7b6e7d: Status 404 returned error can't find the container with id 101b0c845d2a73ee49a8c2ace70a9c3d12c48e5330f2dd89221109b2ef7b6e7d
Oct 30 19:19:04.981630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.981548    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:19:04.981984 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.981716    5033 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:19:04.986493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:04.986467    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b7c9l"
Oct 30 19:19:05.309465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:05.309381    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6rzj" event={"ID":"12011250-d65c-4fa7-9529-bd4731595e41","Type":"ContainerStarted","Data":"d75d0500cd94cafd308d54625c4a88f4e02592e3341d786de3020e95cad9500f"}
Oct 30 19:19:05.309465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:05.309445    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6rzj" event={"ID":"12011250-d65c-4fa7-9529-bd4731595e41","Type":"ContainerStarted","Data":"1d3157a32b627012fae677f165a62153fb3a3ad1f4ce68cb0423a7d872881317"}
Oct 30 19:19:05.311019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:05.310990    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp" event={"ID":"72965977-109a-4e75-b0f5-31f94e98c42a","Type":"ContainerStarted","Data":"6d57c374e390e05eb4e47cc94ddb47d70f895b4f20ec3f46707b51ba9c893e80"}
Oct 30 19:19:05.311125 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:05.311040    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp" event={"ID":"72965977-109a-4e75-b0f5-31f94e98c42a","Type":"ContainerStarted","Data":"101b0c845d2a73ee49a8c2ace70a9c3d12c48e5330f2dd89221109b2ef7b6e7d"}
Oct 30 19:19:05.312204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:05.312179    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl" event={"ID":"962ae636-bcad-4264-9b99-b47896c634e1","Type":"ContainerStarted","Data":"f26f77fb3d6512a2ff71dc7aed12994befe6c979e70f7b0430c581dd6e4393a1"}
Oct 30 19:19:06.237733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:06.237700    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:06.240662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:06.240641    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:06.320381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:06.320347    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-wtwxl" event={"ID":"962ae636-bcad-4264-9b99-b47896c634e1","Type":"ContainerStarted","Data":"b1686982b4b5cd09ff0ee114a31b5a1a40561beda8ee0da2f8b663b0662f2e8e"}
Oct 30 19:19:06.608654 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:06.608631    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:06.609960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:06.609941    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d5c44fb6c-bc7gp"
Oct 30 19:19:07.326072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:07.326034    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6rzj" event={"ID":"12011250-d65c-4fa7-9529-bd4731595e41","Type":"ContainerStarted","Data":"6c06b96e09ae5439b2e393ba520a1156caa62aa0e28bf8fe5d72de70f32d3105"}
Oct 30 19:19:08.055932 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:08.055900    5033 patch_prober.go:29] interesting pod/image-registry-648766b4bf-fs66r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 19:19:08.056092 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:08.055965    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" podUID="172146e6-bd62-4407-93fc-19dc16979806" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:08.087977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:08.087951    5033 patch_prober.go:29] interesting pod/image-registry-5d4cb9956d-hz6xt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 19:19:08.088109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:08.087989    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:09.958556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.958524    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-fkvb6-58b59"]
Oct 30 19:19:09.963365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.963340    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:09.977909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.977883    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"]
Oct 30 19:19:09.987567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.987495    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-fkvb6-58b59"]
Oct 30 19:19:09.987789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.987613    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:09.993187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:09.993162    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"]
Oct 30 19:19:10.008565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.008535    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:19:10.009336 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.009301    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 19:19:10.009604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.009586    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5rcqj\""
Oct 30 19:19:10.073250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.073211    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-njm2t"]
Oct 30 19:19:10.076718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.076669    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.086812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.086789    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dc79d54f-wjdcb"]
Oct 30 19:19:10.089986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.089966    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.096376 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.096352    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.096472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.096395    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b109aa02-9164-47d5-8c6a-704935fea276-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-27fpf\" (UID: \"b109aa02-9164-47d5-8c6a-704935fea276\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:10.096472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.096429    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lct6c\" (UniqueName: \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.096561 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.096528    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.096602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.096591    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.100366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.100343    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:19:10.100446 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.100413    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:19:10.116321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.116296    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-xl6qr\""
Oct 30 19:19:10.117500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.117408    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-njm2t"]
Oct 30 19:19:10.119204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.119181    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc79d54f-wjdcb"]
Oct 30 19:19:10.197137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197106    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-ca-trust-extracted\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197152    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.197302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197178    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7jlq\" (UniqueName: \"kubernetes.io/projected/cadcee6d-8c2f-46ad-8f21-8adb13874d33-kube-api-access-g7jlq\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.197302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197270    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-certificates\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197294    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cadcee6d-8c2f-46ad-8f21-8adb13874d33-tmp\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.197496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197316    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-installation-pull-secrets\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197333    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-bound-sa-token\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197430    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-image-registry-private-configuration\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197595 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197509    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.197595 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197528    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-trusted-ca\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197595 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197533    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.197595 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197554    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b109aa02-9164-47d5-8c6a-704935fea276-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-27fpf\" (UID: \"b109aa02-9164-47d5-8c6a-704935fea276\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:10.197595 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197592    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lct6c\" (UniqueName: \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.197837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197619    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.197837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197637    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-tls\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.197837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.197652    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgj7g\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-kube-api-access-bgj7g\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.198284 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.198259    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.198496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.198475    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.199874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.199856    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b109aa02-9164-47d5-8c6a-704935fea276-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-27fpf\" (UID: \"b109aa02-9164-47d5-8c6a-704935fea276\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:10.208807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.208758    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lct6c\" (UniqueName: \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\") pod \"periodic-gathering-fkvb6-58b59\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") " pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.286581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.286553    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:19:10.287166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.287145    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nmrtr"
Oct 30 19:19:10.299220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299194    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cadcee6d-8c2f-46ad-8f21-8adb13874d33-tmp\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.299342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299228    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-installation-pull-secrets\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299253    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-bound-sa-token\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299282    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-image-registry-private-configuration\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299308    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-trusted-ca\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299336    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-tls\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299355    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgj7g\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-kube-api-access-bgj7g\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299389    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-ca-trust-extracted\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.299604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299418    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7jlq\" (UniqueName: \"kubernetes.io/projected/cadcee6d-8c2f-46ad-8f21-8adb13874d33-kube-api-access-g7jlq\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.299604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.299443    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-certificates\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.300132 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.300111    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cadcee6d-8c2f-46ad-8f21-8adb13874d33-tmp\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.300338 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.300320    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-certificates\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.300637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.300616    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-ca-trust-extracted\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.301980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.301956    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-trusted-ca\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.302113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.302093    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:10.306163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.304427    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-installation-pull-secrets\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.306163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.304530    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-registry-tls\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.306330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.306221    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-image-registry-private-configuration\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.319516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.319495    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgj7g\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-kube-api-access-bgj7g\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.320011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.319988    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49b322c5-f1df-4b2f-bf50-e8f6dfc46493-bound-sa-token\") pod \"image-registry-7dc79d54f-wjdcb\" (UID: \"49b322c5-f1df-4b2f-bf50-e8f6dfc46493\") " pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.321555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.321521    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7jlq\" (UniqueName: \"kubernetes.io/projected/cadcee6d-8c2f-46ad-8f21-8adb13874d33-kube-api-access-g7jlq\") pod \"downloads-6dd789c8b7-njm2t\" (UID: \"cadcee6d-8c2f-46ad-8f21-8adb13874d33\") " pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.389917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.389638    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:10.402014 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.401988    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:10.594009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.593975    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-fkvb6-58b59"]
Oct 30 19:19:10.596939 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:19:10.596910    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9039dbb_e81d_432b_8b47_0c51777dbc6f.slice/crio-550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a WatchSource:0}: Error finding container 550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a: Status 404 returned error can't find the container with id 550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a
Oct 30 19:19:10.617220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.617197    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"]
Oct 30 19:19:10.620484 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:19:10.620457    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb109aa02_9164_47d5_8c6a_704935fea276.slice/crio-2ebad11fd16ffc7fbfc4717877e473dcccd26cb421f5167b45cb4c0d666857d4 WatchSource:0}: Error finding container 2ebad11fd16ffc7fbfc4717877e473dcccd26cb421f5167b45cb4c0d666857d4: Status 404 returned error can't find the container with id 2ebad11fd16ffc7fbfc4717877e473dcccd26cb421f5167b45cb4c0d666857d4
Oct 30 19:19:10.690875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.690837    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-njm2t"]
Oct 30 19:19:10.726773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:10.724499    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc79d54f-wjdcb"]
Oct 30 19:19:11.357758 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.357646    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-njm2t" event={"ID":"cadcee6d-8c2f-46ad-8f21-8adb13874d33","Type":"ContainerStarted","Data":"63f6ce2594a0384f00846fa14cb58e403dfff2a25b822bcc774a415e195ba312"}
Oct 30 19:19:11.360525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.360495    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb" event={"ID":"49b322c5-f1df-4b2f-bf50-e8f6dfc46493","Type":"ContainerStarted","Data":"b9b5f9946a96f5b854fec1439edd06b5b1e0096c69fc9d3a3e9387de8b19bb67"}
Oct 30 19:19:11.360645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.360543    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb" event={"ID":"49b322c5-f1df-4b2f-bf50-e8f6dfc46493","Type":"ContainerStarted","Data":"f3c883797136860deb781d86dfa983d4e9f5e9e3b65c16fee267cae0c9955877"}
Oct 30 19:19:11.360645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.360590    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:11.363474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.363441    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf" event={"ID":"b109aa02-9164-47d5-8c6a-704935fea276","Type":"ContainerStarted","Data":"2ebad11fd16ffc7fbfc4717877e473dcccd26cb421f5167b45cb4c0d666857d4"}
Oct 30 19:19:11.365833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.365806    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fkvb6-58b59" event={"ID":"a9039dbb-e81d-432b-8b47-0c51777dbc6f","Type":"ContainerStarted","Data":"80af1c750062221eb9d5b2e44c63bc9d799a03a770cb19562545b34ca0ef1035"}
Oct 30 19:19:11.365962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.365856    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fkvb6-58b59" event={"ID":"a9039dbb-e81d-432b-8b47-0c51777dbc6f","Type":"ContainerStarted","Data":"550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a"}
Oct 30 19:19:11.383656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.383616    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb" podStartSLOduration=1.383603782 podStartE2EDuration="1.383603782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:11.383066363 +0000 UTC m=+16.352707202" watchObservedRunningTime="2026-10-30 19:19:11.383603782 +0000 UTC m=+16.353244618"
Oct 30 19:19:11.988300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:11.988230    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmrtr_fe0bce47-ad9b-4cbe-b544-30d75ae93300/dns/0.log"
Oct 30 19:19:12.182465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.182434    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmrtr_fe0bce47-ad9b-4cbe-b544-30d75ae93300/kube-rbac-proxy/0.log"
Oct 30 19:19:12.372500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.372462    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf" event={"ID":"b109aa02-9164-47d5-8c6a-704935fea276","Type":"ContainerStarted","Data":"1b8bbd467de0775750f4d30768bff80e88f6956d6af8c3b1ac8c3cf416c6c346"}
Oct 30 19:19:12.389430 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.389359    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf" podStartSLOduration=1.826811568 podStartE2EDuration="3.389343525s" totalImagesPullingTime="1.562531957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:12.388786465 +0000 UTC m=+17.358427300" watchObservedRunningTime="2026-10-30 19:19:12.389343525 +0000 UTC m=+17.358984360"
Oct 30 19:19:12.390123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.390079    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-fkvb6-58b59" podStartSLOduration=3.390072753 podStartE2EDuration="3.390072753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:11.404059518 +0000 UTC m=+16.373700354" watchObservedRunningTime="2026-10-30 19:19:12.390072753 +0000 UTC m=+17.359713588"
Oct 30 19:19:12.781468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.781444    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ln7fq_07071eae-8f8c-4d40-a3c3-57db59367dfb/dns-node-resolver/0.log"
Oct 30 19:19:12.982443 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:12.982424    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d4cb9956d-hz6xt_5348f947-fe2f-4b4d-ae83-2830d5736837/registry/0.log"
Oct 30 19:19:13.182028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:13.181945    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-648766b4bf-fs66r_172146e6-bd62-4407-93fc-19dc16979806/registry/0.log"
Oct 30 19:19:13.377654 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:13.377569    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:13.381177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:13.381155    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7dc79d54f-wjdcb_49b322c5-f1df-4b2f-bf50-e8f6dfc46493/registry/0.log"
Oct 30 19:19:13.382758 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:13.382737    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-27fpf"
Oct 30 19:19:13.581895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:13.581867    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-85l8w_5895a27a-6712-4344-ac33-f49f30f5e920/node-ca/0.log"
Oct 30 19:19:14.183925 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:14.183887    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"
Oct 30 19:19:14.580764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:14.580715    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pt2v2_cbec4229-cce9-465d-8ff4-c01460c9a1b9/serve-healthcheck-canary/0.log"
Oct 30 19:19:16.365368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:16.365336    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bkpng"
Oct 30 19:19:18.802317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.802285    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zkr48"]
Oct 30 19:19:18.810099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.810067    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.825810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.822007    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:19:18.825810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.822177    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:19:18.825810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.822981    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:19:18.825810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.823078    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j9nbc\""
Oct 30 19:19:18.825810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.823130    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:19:18.995528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995495    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.995528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995534    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.995834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995561    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-tls\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.995834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995819    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-textfile\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.995938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995855    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-wtmp\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.995938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995886    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jnk7\" (UniqueName: \"kubernetes.io/projected/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-kube-api-access-4jnk7\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.996039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.995988    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-sys\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.996039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.996013    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-root\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:18.996039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:18.996029    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-metrics-client-ca\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096473    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-sys\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096508    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-root\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096588    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-sys\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096641    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-metrics-client-ca\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096670    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-root\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096731    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096755    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.096802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096789    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-tls\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096816    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-textfile\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096835    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-wtmp\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.096868    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jnk7\" (UniqueName: \"kubernetes.io/projected/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-kube-api-access-4jnk7\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097298 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.097278    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-metrics-client-ca\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097734 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.097709    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-wtmp\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.097851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.097752    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-textfile\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.098115 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.098063    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.099855 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.099833    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.099944 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.099917    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-node-exporter-tls\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.111003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.110977    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jnk7\" (UniqueName: \"kubernetes.io/projected/d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2-kube-api-access-4jnk7\") pod \"node-exporter-zkr48\" (UID: \"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2\") " pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.140662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.140432    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zkr48"
Oct 30 19:19:19.411410 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.411333    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkr48" event={"ID":"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2","Type":"ContainerStarted","Data":"b1eb49fc8ac6e87a1913b6e686935910d1574c34d85e4191ac9fea751e7381c2"}
Oct 30 19:19:19.468010 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:19.467981    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:19:20.016112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:20.016085    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:19:20.417901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:20.417853    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkr48" event={"ID":"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2","Type":"ContainerStarted","Data":"028b16f28435a483f54ebe3610057097474086198f876b68ab1746c921d56072"}
Oct 30 19:19:21.424661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:21.424619    5033 generic.go:356] "Generic (PLEG): container finished" podID="d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2" containerID="028b16f28435a483f54ebe3610057097474086198f876b68ab1746c921d56072" exitCode=0
Oct 30 19:19:21.425065 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:21.424723    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkr48" event={"ID":"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2","Type":"ContainerDied","Data":"028b16f28435a483f54ebe3610057097474086198f876b68ab1746c921d56072"}
Oct 30 19:19:22.432299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:22.432260    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkr48" event={"ID":"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2","Type":"ContainerStarted","Data":"474232902bc9da4c7cea2ebcae549e5ed8875277089e732daf12880efb6d70d3"}
Oct 30 19:19:22.432727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:22.432344    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkr48" event={"ID":"d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2","Type":"ContainerStarted","Data":"cd3e9f5558d1dcd96f058a68aa34326df3837edaf962a6b0c67dd61b1d151dc7"}
Oct 30 19:19:22.456190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:22.456145    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zkr48" podStartSLOduration=3.2600278 podStartE2EDuration="4.456128922s" totalImagesPullingTime="1.120618612s" totalInitContainerRuntime="75.48251ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:22.453841172 +0000 UTC m=+27.423482023" watchObservedRunningTime="2026-10-30 19:19:22.456128922 +0000 UTC m=+27.425769759"
Oct 30 19:19:22.483946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:22.483921    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s2mwq"
Oct 30 19:19:27.457242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:27.457196    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-njm2t" event={"ID":"cadcee6d-8c2f-46ad-8f21-8adb13874d33","Type":"ContainerStarted","Data":"c1ae3a3909a3451b4e5635ff805ad6c37ea8e773efa25585431331e666bc6d90"}
Oct 30 19:19:27.457621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:27.457418    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:27.458583 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:27.458561    5033 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-njm2t container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused" start-of-body=
Oct 30 19:19:27.458691 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:27.458600    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-njm2t" podUID="cadcee6d-8c2f-46ad-8f21-8adb13874d33" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused"
Oct 30 19:19:27.479274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:27.479232    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-njm2t" podStartSLOduration=0.905983208 podStartE2EDuration="17.479221348s" totalImagesPullingTime="16.57323814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:27.47671261 +0000 UTC m=+32.446353442" watchObservedRunningTime="2026-10-30 19:19:27.479221348 +0000 UTC m=+32.448862180"
Oct 30 19:19:28.468778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:28.468456    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-njm2t"
Oct 30 19:19:30.558812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:30.558782    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-frfms"
Oct 30 19:19:32.378586 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:32.378554    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dc79d54f-wjdcb"
Oct 30 19:19:32.440188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:32.436658    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:19:34.990571 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:19:34.990534    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:19:35.034148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.034085    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerName="registry" containerID="cri-o://740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1" gracePeriod=30
Oct 30 19:19:35.329084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.329055    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:19:35.356968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.356943    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357004    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357051    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357222 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357118    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357222 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357145    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357222 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357172    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357221    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357255    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") pod \"5348f947-fe2f-4b4d-ae83-2830d5736837\" (UID: \"5348f947-fe2f-4b4d-ae83-2830d5736837\") "
Oct 30 19:19:35.357482 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357440    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:35.357563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.357539    5033 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-trusted-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.358708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.358541    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:35.361389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.361359    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "kube-api-access-m6ngh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:35.361514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.361462    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:35.361825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.361710    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:35.363098 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.363026    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:35.363773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.363742    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:35.367121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.367068    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted" pod "5348f947-fe2f-4b4d-ae83-2830d5736837" (UID: "5348f947-fe2f-4b4d-ae83-2830d5736837"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460199    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6ngh\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-kube-api-access-m6ngh\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460237    5033 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-image-registry-private-configuration\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460254    5033 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5348f947-fe2f-4b4d-ae83-2830d5736837-ca-trust-extracted\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460269    5033 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-tls\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460297    5033 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5348f947-fe2f-4b4d-ae83-2830d5736837-installation-pull-secrets\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460312    5033 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5348f947-fe2f-4b4d-ae83-2830d5736837-registry-certificates\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.461626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.460327    5033 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5348f947-fe2f-4b4d-ae83-2830d5736837-bound-sa-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:35.500936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.500903    5033 generic.go:356] "Generic (PLEG): container finished" podID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerID="740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1" exitCode=0
Oct 30 19:19:35.501087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.500980    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt"
Oct 30 19:19:35.501087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.501013    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" event={"ID":"5348f947-fe2f-4b4d-ae83-2830d5736837","Type":"ContainerDied","Data":"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"}
Oct 30 19:19:35.501202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.501093    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d4cb9956d-hz6xt" event={"ID":"5348f947-fe2f-4b4d-ae83-2830d5736837","Type":"ContainerDied","Data":"6f82fd3cb76ea40fb110f44d2816a01d05c3121e432593c301c3d318143d3276"}
Oct 30 19:19:35.501202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.501123    5033 scope.go:122] "RemoveContainer" containerID="740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"
Oct 30 19:19:35.512891 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:19:35.512862    5033 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5348f947_fe2f_4b4d_ae83_2830d5736837.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:19:35.516327 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.516303    5033 scope.go:122] "RemoveContainer" containerID="740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"
Oct 30 19:19:35.516652 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:19:35.516632    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1\": container with ID starting with 740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1 not found: ID does not exist" containerID="740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"
Oct 30 19:19:35.516744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.516663    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1"} err="failed to get container status \"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1\": rpc error: code = NotFound desc = could not find container \"740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1\": container with ID starting with 740e6682583131ad7618a1f6b410ca16a3fcf70c386cb7c36c9e2c9d6719b1e1 not found: ID does not exist"
Oct 30 19:19:35.530115 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.530092    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:19:35.534661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:35.534639    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d4cb9956d-hz6xt"]
Oct 30 19:19:37.166122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:37.166091    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" path="/var/lib/kubelet/pods/5348f947-fe2f-4b4d-ae83-2830d5736837/volumes"
Oct 30 19:19:41.746784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:41.746746    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:19:41.747273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:41.746984    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" podUID="438e3014-0622-4bfc-b7a0-7da9fc14472b" containerName="insights-gathering" containerID="cri-o://0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289" gracePeriod=30
Oct 30 19:19:42.058394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.058366    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:19:42.111699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.111666    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") pod \"438e3014-0622-4bfc-b7a0-7da9fc14472b\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") "
Oct 30 19:19:42.111839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.111712    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") pod \"438e3014-0622-4bfc-b7a0-7da9fc14472b\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") "
Oct 30 19:19:42.111839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.111740    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") pod \"438e3014-0622-4bfc-b7a0-7da9fc14472b\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") "
Oct 30 19:19:42.111839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.111806    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") pod \"438e3014-0622-4bfc-b7a0-7da9fc14472b\" (UID: \"438e3014-0622-4bfc-b7a0-7da9fc14472b\") "
Oct 30 19:19:42.112105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.112032    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path" pod "438e3014-0622-4bfc-b7a0-7da9fc14472b" (UID: "438e3014-0622-4bfc-b7a0-7da9fc14472b"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:19:42.112167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.112115    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle" pod "438e3014-0622-4bfc-b7a0-7da9fc14472b" (UID: "438e3014-0622-4bfc-b7a0-7da9fc14472b"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:42.112203 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.112171    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle" pod "438e3014-0622-4bfc-b7a0-7da9fc14472b" (UID: "438e3014-0622-4bfc-b7a0-7da9fc14472b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:42.113905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.113886    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl" pod "438e3014-0622-4bfc-b7a0-7da9fc14472b" (UID: "438e3014-0622-4bfc-b7a0-7da9fc14472b"). InnerVolumeSpecName "kube-api-access-2vdrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:42.212624 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.212598    5033 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/438e3014-0622-4bfc-b7a0-7da9fc14472b-archives-path\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:42.212747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.212623    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vdrl\" (UniqueName: \"kubernetes.io/projected/438e3014-0622-4bfc-b7a0-7da9fc14472b-kube-api-access-2vdrl\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:42.212747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.212664    5033 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-service-ca-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:42.212747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.212703    5033 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438e3014-0622-4bfc-b7a0-7da9fc14472b-trusted-ca-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:42.533621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.533588    5033 generic.go:356] "Generic (PLEG): container finished" podID="438e3014-0622-4bfc-b7a0-7da9fc14472b" containerID="0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289" exitCode=2
Oct 30 19:19:42.533783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.533636    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7"
Oct 30 19:19:42.533783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.533689    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" event={"ID":"438e3014-0622-4bfc-b7a0-7da9fc14472b","Type":"ContainerDied","Data":"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"}
Oct 30 19:19:42.533783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.533734    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v6fdz-f5wz7" event={"ID":"438e3014-0622-4bfc-b7a0-7da9fc14472b","Type":"ContainerDied","Data":"93e5d87d97d2c039538170021c792b1fe73900d1f714ffc89a4da842fcc65b5f"}
Oct 30 19:19:42.533783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.533755    5033 scope.go:122] "RemoveContainer" containerID="0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"
Oct 30 19:19:42.542253 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.542234    5033 scope.go:122] "RemoveContainer" containerID="0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"
Oct 30 19:19:42.542500 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:19:42.542478    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289\": container with ID starting with 0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289 not found: ID does not exist" containerID="0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"
Oct 30 19:19:42.542558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.542512    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289"} err="failed to get container status \"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289\": rpc error: code = NotFound desc = could not find container \"0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289\": container with ID starting with 0d019124ab6d730a5551462e2c3d44cc5965f9eb8d2c0995d5604c51c6224289 not found: ID does not exist"
Oct 30 19:19:42.555593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.555571    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:19:42.557747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:42.557726    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-v6fdz-f5wz7"]
Oct 30 19:19:43.166430 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:43.166392    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="438e3014-0622-4bfc-b7a0-7da9fc14472b" path="/var/lib/kubelet/pods/438e3014-0622-4bfc-b7a0-7da9fc14472b/volumes"
Oct 30 19:19:57.507183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.507128    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" podUID="172146e6-bd62-4407-93fc-19dc16979806" containerName="registry" containerID="cri-o://c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385" gracePeriod=30
Oct 30 19:19:57.779493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.779435    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:19:57.824851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.824824    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.824982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.824862    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.824982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.824897    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825004    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825033    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825063    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825094    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825152    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") pod \"172146e6-bd62-4407-93fc-19dc16979806\" (UID: \"172146e6-bd62-4407-93fc-19dc16979806\") "
Oct 30 19:19:57.825471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825419    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:57.825733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.825709    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:57.827554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.827504    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:57.827554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.827535    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:57.827861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.827839    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:57.827926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.827886    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "kube-api-access-zglrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:57.828217 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.828202    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:57.832477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.832456    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted" pod "172146e6-bd62-4407-93fc-19dc16979806" (UID: "172146e6-bd62-4407-93fc-19dc16979806"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:19:57.926175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926154    5033 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-trusted-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926174    5033 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/172146e6-bd62-4407-93fc-19dc16979806-registry-certificates\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926184    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zglrx\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-kube-api-access-zglrx\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926193    5033 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-installation-pull-secrets\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926202    5033 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/172146e6-bd62-4407-93fc-19dc16979806-ca-trust-extracted\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926210    5033 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-registry-tls\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926219    5033 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/172146e6-bd62-4407-93fc-19dc16979806-bound-sa-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:57.926303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:57.926229    5033 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/172146e6-bd62-4407-93fc-19dc16979806-image-registry-private-configuration\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:58.598239 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.598210    5033 generic.go:356] "Generic (PLEG): container finished" podID="172146e6-bd62-4407-93fc-19dc16979806" containerID="c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385" exitCode=0
Oct 30 19:19:58.598631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.598268    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648766b4bf-fs66r"
Oct 30 19:19:58.598631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.598294    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" event={"ID":"172146e6-bd62-4407-93fc-19dc16979806","Type":"ContainerDied","Data":"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"}
Oct 30 19:19:58.598631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.598359    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648766b4bf-fs66r" event={"ID":"172146e6-bd62-4407-93fc-19dc16979806","Type":"ContainerDied","Data":"76561b593be4efb295fc15ebb3f2292691d6482c26cf0e6d5b89b8b18d3dc065"}
Oct 30 19:19:58.598631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.598377    5033 scope.go:122] "RemoveContainer" containerID="c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"
Oct 30 19:19:58.608188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.608170    5033 scope.go:122] "RemoveContainer" containerID="c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"
Oct 30 19:19:58.608441 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:19:58.608423    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385\": container with ID starting with c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385 not found: ID does not exist" containerID="c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"
Oct 30 19:19:58.608488 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.608449    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385"} err="failed to get container status \"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385\": rpc error: code = NotFound desc = could not find container \"c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385\": container with ID starting with c12b96dea845f26569d97a626a09f8e5e78f4407d435aab8732c8a5119dfa385 not found: ID does not exist"
Oct 30 19:19:58.624380 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.624357    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:19:58.633604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:58.633585    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-648766b4bf-fs66r"]
Oct 30 19:19:59.166535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:19:59.166507    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="172146e6-bd62-4407-93fc-19dc16979806" path="/var/lib/kubelet/pods/172146e6-bd62-4407-93fc-19dc16979806/volumes"
Oct 30 19:20:04.985191 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:20:04.985156    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:20:09.643584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:09.643553    5033 generic.go:356] "Generic (PLEG): container finished" podID="4b3670eb-d511-4036-bb90-5fed8a4320b9" containerID="bea5f2825ea788abdd2b2cce4e80e1997a008e826aaffbca92755a161e5f0963" exitCode=0
Oct 30 19:20:09.643985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:09.643635    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2" event={"ID":"4b3670eb-d511-4036-bb90-5fed8a4320b9","Type":"ContainerDied","Data":"bea5f2825ea788abdd2b2cce4e80e1997a008e826aaffbca92755a161e5f0963"}
Oct 30 19:20:09.644027 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:09.643989    5033 scope.go:122] "RemoveContainer" containerID="bea5f2825ea788abdd2b2cce4e80e1997a008e826aaffbca92755a161e5f0963"
Oct 30 19:20:10.649168 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:10.649128    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgws2" event={"ID":"4b3670eb-d511-4036-bb90-5fed8a4320b9","Type":"ContainerStarted","Data":"99a505d9d3ce0e9614997ac5a8c492386be51ad4159be02763797251000a2fca"}
Oct 30 19:20:34.985602 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:20:34.985564    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:20:36.749230 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:36.749196    5033 generic.go:356] "Generic (PLEG): container finished" podID="a9039dbb-e81d-432b-8b47-0c51777dbc6f" containerID="80af1c750062221eb9d5b2e44c63bc9d799a03a770cb19562545b34ca0ef1035" exitCode=0
Oct 30 19:20:36.749671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:36.749263    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fkvb6-58b59" event={"ID":"a9039dbb-e81d-432b-8b47-0c51777dbc6f","Type":"ContainerDied","Data":"80af1c750062221eb9d5b2e44c63bc9d799a03a770cb19562545b34ca0ef1035"}
Oct 30 19:20:37.877517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:37.877494    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:20:38.021244 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021179    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\") pod \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") "
Oct 30 19:20:38.021452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021246    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\" (UniqueName: \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\") pod \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") "
Oct 30 19:20:38.021452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021273    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\") pod \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") "
Oct 30 19:20:38.021452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021318    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\") pod \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\" (UID: \"a9039dbb-e81d-432b-8b47-0c51777dbc6f\") "
Oct 30 19:20:38.021638 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021528    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle" pod "a9039dbb-e81d-432b-8b47-0c51777dbc6f" (UID: "a9039dbb-e81d-432b-8b47-0c51777dbc6f"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:38.021723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021641    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path" pod "a9039dbb-e81d-432b-8b47-0c51777dbc6f" (UID: "a9039dbb-e81d-432b-8b47-0c51777dbc6f"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:38.021723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.021669    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle" pod "a9039dbb-e81d-432b-8b47-0c51777dbc6f" (UID: "a9039dbb-e81d-432b-8b47-0c51777dbc6f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:38.023243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.023224    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c" pod "a9039dbb-e81d-432b-8b47-0c51777dbc6f" (UID: "a9039dbb-e81d-432b-8b47-0c51777dbc6f"). InnerVolumeSpecName "kube-api-access-lct6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:38.121843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.121822    5033 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-trusted-ca-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:38.121843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.121841    5033 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9039dbb-e81d-432b-8b47-0c51777dbc6f-archives-path\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:38.121961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.121851    5033 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9039dbb-e81d-432b-8b47-0c51777dbc6f-service-ca-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:38.121961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.121859    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lct6c\" (UniqueName: \"kubernetes.io/projected/a9039dbb-e81d-432b-8b47-0c51777dbc6f-kube-api-access-lct6c\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:38.757797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.757765    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fkvb6-58b59" event={"ID":"a9039dbb-e81d-432b-8b47-0c51777dbc6f","Type":"ContainerDied","Data":"550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a"}
Oct 30 19:20:38.757935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.757808    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fkvb6-58b59"
Oct 30 19:20:38.757935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:20:38.757818    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="550e6bd3b64ba966df43b2d8f28d054dcb5c95e61560cd2ea44d035fd04acd3a"
Oct 30 19:21:04.984693 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:21:04.984643    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:21:34.987513 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:21:34.987473    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:21:35.947969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.947933    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"]
Oct 30 19:21:35.948443 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948424    5033 cpu_manager.go:414] "containerMap: removing container" podUID="172146e6-bd62-4407-93fc-19dc16979806" containerName="registry"
Oct 30 19:21:35.948443 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948443    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="172146e6-bd62-4407-93fc-19dc16979806" containerName="registry"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948463    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerName="registry"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948471    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerName="registry"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948508    5033 cpu_manager.go:414] "containerMap: removing container" podUID="a9039dbb-e81d-432b-8b47-0c51777dbc6f" containerName="insights-gathering"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948517    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9039dbb-e81d-432b-8b47-0c51777dbc6f" containerName="insights-gathering"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948537    5033 cpu_manager.go:414] "containerMap: removing container" podUID="438e3014-0622-4bfc-b7a0-7da9fc14472b" containerName="insights-gathering"
Oct 30 19:21:35.948594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948545    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="438e3014-0622-4bfc-b7a0-7da9fc14472b" containerName="insights-gathering"
Oct 30 19:21:35.948810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948623    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9039dbb-e81d-432b-8b47-0c51777dbc6f" containerName="insights-gathering"
Oct 30 19:21:35.948810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948637    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="438e3014-0622-4bfc-b7a0-7da9fc14472b" containerName="insights-gathering"
Oct 30 19:21:35.948810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948646    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="172146e6-bd62-4407-93fc-19dc16979806" containerName="registry"
Oct 30 19:21:35.948810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.948655    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5348f947-fe2f-4b4d-ae83-2830d5736837" containerName="registry"
Oct 30 19:21:35.956434 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.956413    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:35.967476 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.967450    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-f7dvq\""
Oct 30 19:21:35.968924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.968861    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"]
Oct 30 19:21:35.976088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:35.976071    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 19:21:36.011508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011482    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e0d197-586c-47f8-a13d-693cd930ac03-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011515    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011545    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011566    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011668    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnccl\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-kube-api-access-lnccl\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011716    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011752    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011800    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.011892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.011845    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.112990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.112965    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.112996    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnccl\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-kube-api-access-lnccl\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113017    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113058    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113084    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113111    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113138    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e0d197-586c-47f8-a13d-693cd930ac03-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113169    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113219    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113491    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113536    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113602    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113624    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.113905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.113883    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e0d197-586c-47f8-a13d-693cd930ac03-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.115518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.115497    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.115742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.115723    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.121926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.121910    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnccl\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-kube-api-access-lnccl\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.122087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.122068    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e0d197-586c-47f8-a13d-693cd930ac03-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g\" (UID: \"c6e0d197-586c-47f8-a13d-693cd930ac03\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.271969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.271947    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:36.543741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.543713    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"]
Oct 30 19:21:36.545805 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:21:36.545777    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e0d197_586c_47f8_a13d_693cd930ac03.slice/crio-91646cf2a6da4ea9b5cd531415152b8f58dbcbac0eb49746b4cbc42a459c95bb WatchSource:0}: Error finding container 91646cf2a6da4ea9b5cd531415152b8f58dbcbac0eb49746b4cbc42a459c95bb: Status 404 returned error can't find the container with id 91646cf2a6da4ea9b5cd531415152b8f58dbcbac0eb49746b4cbc42a459c95bb
Oct 30 19:21:36.977839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:36.977808    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g" event={"ID":"c6e0d197-586c-47f8-a13d-693cd930ac03","Type":"ContainerStarted","Data":"91646cf2a6da4ea9b5cd531415152b8f58dbcbac0eb49746b4cbc42a459c95bb"}
Oct 30 19:21:37.106179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.106147    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"]
Oct 30 19:21:37.108934 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.108918    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.118190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.118164    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:21:37.118392 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.118370    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:21:37.118540 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.118521    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-bhl57\""
Oct 30 19:21:37.119073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.119054    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"]
Oct 30 19:21:37.222869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.222843    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzjzv\" (UniqueName: \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.222993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.222894    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.222993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.222910    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.323923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.323871    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzjzv\" (UniqueName: \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.323923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.323907    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.323923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.323924    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.324221 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.324202    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.324268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.324237    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.334175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.334156    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzjzv\" (UniqueName: \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.421971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.421952    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:21:37.688899 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.688876    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"]
Oct 30 19:21:37.690768 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:21:37.690746    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3caabaf9_ae67_4a9b_9704_de01d089e607.slice/crio-0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1 WatchSource:0}: Error finding container 0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1: Status 404 returned error can't find the container with id 0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1
Oct 30 19:21:37.984637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:37.984601    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2" event={"ID":"3caabaf9-ae67-4a9b-9704-de01d089e607","Type":"ContainerStarted","Data":"0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1"}
Oct 30 19:21:39.742746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.742709    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"]
Oct 30 19:21:39.749296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.749273    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.756802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.756781    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"]
Oct 30 19:21:39.759483 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.759462    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-x46q9\""
Oct 30 19:21:39.784693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.783597    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"]
Oct 30 19:21:39.788109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.788036    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.793111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.793085    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"]
Oct 30 19:21:39.824585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.824551    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"]
Oct 30 19:21:39.827048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.827012    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:39.837097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.837075    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"]
Oct 30 19:21:39.847703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847658    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.847806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847725    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.847866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847841    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.847926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847877    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szxcq\" (UniqueName: \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.847926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847895    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvdbf\" (UniqueName: \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.848025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.847953    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.858346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.858324    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-kstl2\""
Oct 30 19:21:39.858478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.858453    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-6xbps\""
Oct 30 19:21:39.900716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.900663    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:39.900847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.900758    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:39.900847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.900802    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:39.948459 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948434    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.948569 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948501    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.948645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948619    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2kv4\" (UniqueName: \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:39.948767 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948697    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.948767 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948726    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:39.948880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948796    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szxcq\" (UniqueName: \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.948880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948801    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.948880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948823    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdbf\" (UniqueName: \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.948880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948867    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.949036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.948920    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:39.949036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.949013    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.949112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.949037    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.949429 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.949406    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.958306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.958287    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdbf\" (UniqueName: \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:39.958572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.958550    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szxcq\" (UniqueName: \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:39.995390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:39.995334    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g" event={"ID":"c6e0d197-586c-47f8-a13d-693cd930ac03","Type":"ContainerStarted","Data":"c67e27d630f8fda511a5e3f475c9752a4f3b8105d4b1cdafe0c5fbfe05d6864c"}
Oct 30 19:21:40.020509 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.020375    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g" podStartSLOduration=1.6681365700000002 podStartE2EDuration="5.020356345s" totalImagesPullingTime="3.352219775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:40.019844292 +0000 UTC m=+164.989485188" watchObservedRunningTime="2026-10-30 19:21:40.020356345 +0000 UTC m=+164.989997181"
Oct 30 19:21:40.049605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.049574    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2kv4\" (UniqueName: \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.049605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.049608    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.049834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.049747    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.050041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.050019    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.050195 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.050171    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.059289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.059268    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2kv4\" (UniqueName: \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.071118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.071087    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:40.105745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.105721    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:40.144982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.144201    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:40.272335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.272274    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:40.276985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.276964    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:40.354952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.354927    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"]
Oct 30 19:21:40.357724 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:21:40.357697    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79c5438_2d88_4e03_ab8c_76a6dac50d27.slice/crio-a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae WatchSource:0}: Error finding container a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae: Status 404 returned error can't find the container with id a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae
Oct 30 19:21:40.398866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.397223    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"]
Oct 30 19:21:40.400819 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:21:40.400789    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc06b9935_723e_456b_985a_17c63c280c05.slice/crio-10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a WatchSource:0}: Error finding container 10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a: Status 404 returned error can't find the container with id 10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a
Oct 30 19:21:40.434828 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:40.434800    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"]
Oct 30 19:21:40.438056 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:21:40.438024    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de58a25_facd_4f54_ac10_d959335d5ba6.slice/crio-c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6 WatchSource:0}: Error finding container c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6: Status 404 returned error can't find the container with id c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6
Oct 30 19:21:41.003396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:41.003316    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9" event={"ID":"7de58a25-facd-4f54-ac10-d959335d5ba6","Type":"ContainerStarted","Data":"c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6"}
Oct 30 19:21:41.005844 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:41.005814    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l" event={"ID":"c06b9935-723e-456b-985a-17c63c280c05","Type":"ContainerStarted","Data":"10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a"}
Oct 30 19:21:41.009066 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:41.009039    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2" event={"ID":"f79c5438-2d88-4e03-ab8c-76a6dac50d27","Type":"ContainerStarted","Data":"a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae"}
Oct 30 19:21:41.010009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:41.009432    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:41.010777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:41.010494    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g"
Oct 30 19:21:46.044508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.044429    5033 generic.go:356] "Generic (PLEG): container finished" podID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerID="73f90b80dd1bed251c6282e8a7e6f0a83db847dd4dccac275942bf9c656998e4" exitCode=0
Oct 30 19:21:46.044918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.044515    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2" event={"ID":"3caabaf9-ae67-4a9b-9704-de01d089e607","Type":"ContainerDied","Data":"73f90b80dd1bed251c6282e8a7e6f0a83db847dd4dccac275942bf9c656998e4"}
Oct 30 19:21:46.046052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.046027    5033 generic.go:356] "Generic (PLEG): container finished" podID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerID="4fcf2db15b13262455ca568cfd8e4f27fb9e754bda35920aafbe2b4d3126ec78" exitCode=0
Oct 30 19:21:46.046157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.046105    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9" event={"ID":"7de58a25-facd-4f54-ac10-d959335d5ba6","Type":"ContainerDied","Data":"4fcf2db15b13262455ca568cfd8e4f27fb9e754bda35920aafbe2b4d3126ec78"}
Oct 30 19:21:46.047699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.047661    5033 generic.go:356] "Generic (PLEG): container finished" podID="c06b9935-723e-456b-985a-17c63c280c05" containerID="b63e73786e57ac5889d71122ab6deb2479aedb701e34a285384b99ceacfddc6c" exitCode=0
Oct 30 19:21:46.047796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.047696    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l" event={"ID":"c06b9935-723e-456b-985a-17c63c280c05","Type":"ContainerDied","Data":"b63e73786e57ac5889d71122ab6deb2479aedb701e34a285384b99ceacfddc6c"}
Oct 30 19:21:46.049233 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.049210    5033 generic.go:356] "Generic (PLEG): container finished" podID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerID="7aaae47aed76a39366f9980dfda185097cfcb5686e0160e61ffc7243858bea1d" exitCode=0
Oct 30 19:21:46.049314 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:46.049281    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2" event={"ID":"f79c5438-2d88-4e03-ab8c-76a6dac50d27","Type":"ContainerDied","Data":"7aaae47aed76a39366f9980dfda185097cfcb5686e0160e61ffc7243858bea1d"}
Oct 30 19:21:48.061358 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.061323    5033 generic.go:356] "Generic (PLEG): container finished" podID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerID="1fd7b43970b583b5125b919b9d081e986f9b1f5c440bc76a34f693ba811f7a9b" exitCode=0
Oct 30 19:21:48.061838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.061412    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9" event={"ID":"7de58a25-facd-4f54-ac10-d959335d5ba6","Type":"ContainerDied","Data":"1fd7b43970b583b5125b919b9d081e986f9b1f5c440bc76a34f693ba811f7a9b"}
Oct 30 19:21:48.063506 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.063491    5033 generic.go:356] "Generic (PLEG): container finished" podID="c06b9935-723e-456b-985a-17c63c280c05" containerID="2ab523bc1b9edf22e3fa1cf9f7e28d08783d37343ba29b90c51ccc20a598cb9d" exitCode=0
Oct 30 19:21:48.063762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.063722    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l" event={"ID":"c06b9935-723e-456b-985a-17c63c280c05","Type":"ContainerDied","Data":"2ab523bc1b9edf22e3fa1cf9f7e28d08783d37343ba29b90c51ccc20a598cb9d"}
Oct 30 19:21:48.065789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.065770    5033 generic.go:356] "Generic (PLEG): container finished" podID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerID="d3b1cdae93f5d25b60315f4cf925db52761c452670dd84655e07b5ab39864897" exitCode=0
Oct 30 19:21:48.065880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:48.065802    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2" event={"ID":"f79c5438-2d88-4e03-ab8c-76a6dac50d27","Type":"ContainerDied","Data":"d3b1cdae93f5d25b60315f4cf925db52761c452670dd84655e07b5ab39864897"}
Oct 30 19:21:49.074062 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:49.074027    5033 generic.go:356] "Generic (PLEG): container finished" podID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerID="2376d9f7701f08570a77d02ad4eb6afd626337e536b92c5f582d09642022550e" exitCode=0
Oct 30 19:21:49.074499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:49.074075    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2" event={"ID":"3caabaf9-ae67-4a9b-9704-de01d089e607","Type":"ContainerDied","Data":"2376d9f7701f08570a77d02ad4eb6afd626337e536b92c5f582d09642022550e"}
Oct 30 19:21:58.131123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.131086    5033 generic.go:356] "Generic (PLEG): container finished" podID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerID="7b4956697c24337d11195dd6ec9a4906acfebea2e69fae1efcb5f99c4393bfb6" exitCode=0
Oct 30 19:21:58.131609 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.131159    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9" event={"ID":"7de58a25-facd-4f54-ac10-d959335d5ba6","Type":"ContainerDied","Data":"7b4956697c24337d11195dd6ec9a4906acfebea2e69fae1efcb5f99c4393bfb6"}
Oct 30 19:21:58.133288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.133261    5033 generic.go:356] "Generic (PLEG): container finished" podID="c06b9935-723e-456b-985a-17c63c280c05" containerID="857ce6c9ec4e581992e16ea1ca44b4d0811f811d5c734f36b234e5bcaab5ab5d" exitCode=0
Oct 30 19:21:58.133424 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.133333    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l" event={"ID":"c06b9935-723e-456b-985a-17c63c280c05","Type":"ContainerDied","Data":"857ce6c9ec4e581992e16ea1ca44b4d0811f811d5c734f36b234e5bcaab5ab5d"}
Oct 30 19:21:58.135272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.135252    5033 generic.go:356] "Generic (PLEG): container finished" podID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerID="1e42a0c6ea9d9bad6f71f43ae65d6497041035c81e4605e8b2668a6f34006e06" exitCode=0
Oct 30 19:21:58.136354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:58.135328    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2" event={"ID":"f79c5438-2d88-4e03-ab8c-76a6dac50d27","Type":"ContainerDied","Data":"1e42a0c6ea9d9bad6f71f43ae65d6497041035c81e4605e8b2668a6f34006e06"}
Oct 30 19:21:59.141998 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.141964    5033 generic.go:356] "Generic (PLEG): container finished" podID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerID="65debd0fa749329a387113679348548fa781f7567cae5fa7fa7743c3c3334fda" exitCode=0
Oct 30 19:21:59.142436 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.142047    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2" event={"ID":"3caabaf9-ae67-4a9b-9704-de01d089e607","Type":"ContainerDied","Data":"65debd0fa749329a387113679348548fa781f7567cae5fa7fa7743c3c3334fda"}
Oct 30 19:21:59.299638 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.299615    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:21:59.303002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.302982    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:21:59.326069 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.326045    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:21:59.427454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427396    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\" (UniqueName: \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\") pod \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") "
Oct 30 19:21:59.427454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427425    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\" (UniqueName: \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\") pod \"c06b9935-723e-456b-985a-17c63c280c05\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") "
Oct 30 19:21:59.427601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427475    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\") pod \"7de58a25-facd-4f54-ac10-d959335d5ba6\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") "
Oct 30 19:21:59.427601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427494    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\") pod \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") "
Oct 30 19:21:59.427601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427526    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\") pod \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\" (UID: \"f79c5438-2d88-4e03-ab8c-76a6dac50d27\") "
Oct 30 19:21:59.427601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427554    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\") pod \"c06b9935-723e-456b-985a-17c63c280c05\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") "
Oct 30 19:21:59.427846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427599    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\" (UniqueName: \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\") pod \"7de58a25-facd-4f54-ac10-d959335d5ba6\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") "
Oct 30 19:21:59.427846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.427768    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\") pod \"c06b9935-723e-456b-985a-17c63c280c05\" (UID: \"c06b9935-723e-456b-985a-17c63c280c05\") "
Oct 30 19:21:59.428089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.428069    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\") pod \"7de58a25-facd-4f54-ac10-d959335d5ba6\" (UID: \"7de58a25-facd-4f54-ac10-d959335d5ba6\") "
Oct 30 19:21:59.428159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.428114    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle" pod "f79c5438-2d88-4e03-ab8c-76a6dac50d27" (UID: "f79c5438-2d88-4e03-ab8c-76a6dac50d27"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.428756 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.428229    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle" pod "c06b9935-723e-456b-985a-17c63c280c05" (UID: "c06b9935-723e-456b-985a-17c63c280c05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.429034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.428885    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle" pod "7de58a25-facd-4f54-ac10-d959335d5ba6" (UID: "7de58a25-facd-4f54-ac10-d959335d5ba6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.429034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.428977    5033 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.429034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.429001    5033 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.429034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.429015    5033 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.430056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.430036    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf" pod "f79c5438-2d88-4e03-ab8c-76a6dac50d27" (UID: "f79c5438-2d88-4e03-ab8c-76a6dac50d27"). InnerVolumeSpecName "kube-api-access-bvdbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:59.430137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.430054    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq" pod "c06b9935-723e-456b-985a-17c63c280c05" (UID: "c06b9935-723e-456b-985a-17c63c280c05"). InnerVolumeSpecName "kube-api-access-szxcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:59.430225 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.430202    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4" pod "7de58a25-facd-4f54-ac10-d959335d5ba6" (UID: "7de58a25-facd-4f54-ac10-d959335d5ba6"). InnerVolumeSpecName "kube-api-access-j2kv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:59.433867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.433841    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util" pod "f79c5438-2d88-4e03-ab8c-76a6dac50d27" (UID: "f79c5438-2d88-4e03-ab8c-76a6dac50d27"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.434212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.434192    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util" pod "c06b9935-723e-456b-985a-17c63c280c05" (UID: "c06b9935-723e-456b-985a-17c63c280c05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.435807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.435789    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util" pod "7de58a25-facd-4f54-ac10-d959335d5ba6" (UID: "7de58a25-facd-4f54-ac10-d959335d5ba6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:59.529723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529699    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2kv4\" (UniqueName: \"kubernetes.io/projected/7de58a25-facd-4f54-ac10-d959335d5ba6-kube-api-access-j2kv4\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.529723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529718    5033 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c06b9935-723e-456b-985a-17c63c280c05-util\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.529831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529727    5033 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7de58a25-facd-4f54-ac10-d959335d5ba6-util\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.529831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529736    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvdbf\" (UniqueName: \"kubernetes.io/projected/f79c5438-2d88-4e03-ab8c-76a6dac50d27-kube-api-access-bvdbf\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.529831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529744    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szxcq\" (UniqueName: \"kubernetes.io/projected/c06b9935-723e-456b-985a-17c63c280c05-kube-api-access-szxcq\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:59.529831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:21:59.529753    5033 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79c5438-2d88-4e03-ab8c-76a6dac50d27-util\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:00.149092 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.149064    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9"
Oct 30 19:22:00.149505 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.149062    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4z6d9" event={"ID":"7de58a25-facd-4f54-ac10-d959335d5ba6","Type":"ContainerDied","Data":"c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6"}
Oct 30 19:22:00.149505 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.149186    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4b140518fc517d7933701670f043fbe6636d86e4c3df9fcf9f51c1d01deb8b6"
Oct 30 19:22:00.151030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.151001    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l" event={"ID":"c06b9935-723e-456b-985a-17c63c280c05","Type":"ContainerDied","Data":"10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a"}
Oct 30 19:22:00.151164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.151054    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10e59793dd0cb69e1925d596ac051dbfb2b6776f5da6c18d1094cdde411d487a"
Oct 30 19:22:00.151164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.151010    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cd8h6l"
Oct 30 19:22:00.152949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.152928    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2"
Oct 30 19:22:00.153077 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.152955    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jflf2" event={"ID":"f79c5438-2d88-4e03-ab8c-76a6dac50d27","Type":"ContainerDied","Data":"a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae"}
Oct 30 19:22:00.153077 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.152988    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a28ff5c595e6a95b93c1ac0bb73f329e69bbe7dd638b5337967c599c005935ae"
Oct 30 19:22:00.276003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.275981    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:22:00.434958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.434899    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\") pod \"3caabaf9-ae67-4a9b-9704-de01d089e607\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") "
Oct 30 19:22:00.435079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.434991    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\") pod \"3caabaf9-ae67-4a9b-9704-de01d089e607\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") "
Oct 30 19:22:00.435079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.435015    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\" (UniqueName: \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\") pod \"3caabaf9-ae67-4a9b-9704-de01d089e607\" (UID: \"3caabaf9-ae67-4a9b-9704-de01d089e607\") "
Oct 30 19:22:00.435334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.435305    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle" pod "3caabaf9-ae67-4a9b-9704-de01d089e607" (UID: "3caabaf9-ae67-4a9b-9704-de01d089e607"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:00.436928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.436907    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv" pod "3caabaf9-ae67-4a9b-9704-de01d089e607" (UID: "3caabaf9-ae67-4a9b-9704-de01d089e607"). InnerVolumeSpecName "kube-api-access-kzjzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:00.440021 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.439997    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util" pod "3caabaf9-ae67-4a9b-9704-de01d089e607" (UID: "3caabaf9-ae67-4a9b-9704-de01d089e607"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:00.536403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.536379    5033 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-util\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:00.536403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.536405    5033 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3caabaf9-ae67-4a9b-9704-de01d089e607-bundle\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:00.536517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:00.536415    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzjzv\" (UniqueName: \"kubernetes.io/projected/3caabaf9-ae67-4a9b-9704-de01d089e607-kube-api-access-kzjzv\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:01.159333 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:01.159306    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2"
Oct 30 19:22:01.159727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:01.159328    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6447wnk2" event={"ID":"3caabaf9-ae67-4a9b-9704-de01d089e607","Type":"ContainerDied","Data":"0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1"}
Oct 30 19:22:01.159727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:01.159368    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0065e8f8d0577c84244b493ae6f6b4a17d30ca1409386f006e379f2edd5280d1"
Oct 30 19:22:04.984986 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:22:04.984952    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:22:12.110262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110229    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"]
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110547    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110559    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110572    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110577    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110585    5033 cpu_manager.go:414] "containerMap: removing container" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110590    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110598    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110602    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110610    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110615    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110627    5033 cpu_manager.go:414] "containerMap: removing container" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110634    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110644    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110650    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110663    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110668    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="pull"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110693    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110698    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110703    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110708    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="util"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110730    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110736    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110743    5033 cpu_manager.go:414] "containerMap: removing container" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="extract"
Oct 30 19:22:12.110744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110747    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="extract"
Oct 30 19:22:12.111730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110815    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="7de58a25-facd-4f54-ac10-d959335d5ba6" containerName="extract"
Oct 30 19:22:12.111730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110825    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="c06b9935-723e-456b-985a-17c63c280c05" containerName="extract"
Oct 30 19:22:12.111730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110832    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3caabaf9-ae67-4a9b-9704-de01d089e607" containerName="extract"
Oct 30 19:22:12.111730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.110840    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="f79c5438-2d88-4e03-ab8c-76a6dac50d27" containerName="extract"
Oct 30 19:22:12.113103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.113083    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116088    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116124    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116148    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116309 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116290    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116325    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.116384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.116357    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h85r\" (UniqueName: \"kubernetes.io/projected/c50d5442-fff2-47ab-8ee9-a6375c2b46df-kube-api-access-2h85r\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.143641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.143619    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"]
Oct 30 19:22:12.197386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.197364    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 19:22:12.197584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.197565    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 19:22:12.197635 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.197624    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 19:22:12.197671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.197630    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-zpjmc\""
Oct 30 19:22:12.200471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.200455    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 19:22:12.200882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.200866    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 19:22:12.217456 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217433    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217545 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217467    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217545 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217506    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h85r\" (UniqueName: \"kubernetes.io/projected/c50d5442-fff2-47ab-8ee9-a6375c2b46df-kube-api-access-2h85r\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217625 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217543    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217625 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217575    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217719    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.217900 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.217883    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-manifests\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.219796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.219775    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c50d5442-fff2-47ab-8ee9-a6375c2b46df-tmp\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.219982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.219964    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-webhook-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.220042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.220013    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-metrics-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.220165 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.220143    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c50d5442-fff2-47ab-8ee9-a6375c2b46df-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.227186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.227163    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h85r\" (UniqueName: \"kubernetes.io/projected/c50d5442-fff2-47ab-8ee9-a6375c2b46df-kube-api-access-2h85r\") pod \"opendatahub-operator-controller-manager-668f68f55d-sbzvc\" (UID: \"c50d5442-fff2-47ab-8ee9-a6375c2b46df\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.426414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.426364    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:12.783813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:12.783784    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"]
Oct 30 19:22:13.212560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:13.212528    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc" event={"ID":"c50d5442-fff2-47ab-8ee9-a6375c2b46df","Type":"ContainerStarted","Data":"adacb599e238d224467cde85371e348aa584ddc257b6c1354feaac47b9f1304b"}
Oct 30 19:22:16.227691 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:16.227640    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc" event={"ID":"c50d5442-fff2-47ab-8ee9-a6375c2b46df","Type":"ContainerStarted","Data":"bb3b4c236dd324dcb91dce25c3d6edc26c851756ee81e5cd68c6bac20bfba83f"}
Oct 30 19:22:17.232606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:17.232570    5033 generic.go:356] "Generic (PLEG): container finished" podID="c50d5442-fff2-47ab-8ee9-a6375c2b46df" containerID="bb3b4c236dd324dcb91dce25c3d6edc26c851756ee81e5cd68c6bac20bfba83f" exitCode=0
Oct 30 19:22:17.233051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:17.232653    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc" event={"ID":"c50d5442-fff2-47ab-8ee9-a6375c2b46df","Type":"ContainerDied","Data":"bb3b4c236dd324dcb91dce25c3d6edc26c851756ee81e5cd68c6bac20bfba83f"}
Oct 30 19:22:18.238762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:18.238726    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc" event={"ID":"c50d5442-fff2-47ab-8ee9-a6375c2b46df","Type":"ContainerStarted","Data":"ba060173028f39ddc108d8e3ada364ac70624c468149348542b55369ac6d2abe"}
Oct 30 19:22:18.239142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:18.238980    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:18.480866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:18.480818    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc" podStartSLOduration=2.653197864 podStartE2EDuration="6.480801288s" totalImagesPullingTime="3.400793821s" totalInitContainerRuntime="426.809603ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:18.474298011 +0000 UTC m=+203.443938857" watchObservedRunningTime="2026-10-30 19:22:18.480801288 +0000 UTC m=+203.450442124"
Oct 30 19:22:20.225006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.224975    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"]
Oct 30 19:22:20.227126 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.227109    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.234821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.234796    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 19:22:20.235133 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.235104    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-khn6j\""
Oct 30 19:22:20.244130 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.244105    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"]
Oct 30 19:22:20.270180 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.270142    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cade7b49-ca45-490c-870d-9c9b9eea59cd-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.270285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.270193    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk9l8\" (UniqueName: \"kubernetes.io/projected/cade7b49-ca45-490c-870d-9c9b9eea59cd-kube-api-access-xk9l8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.334915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.334899    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:22:20.371308 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.371285    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk9l8\" (UniqueName: \"kubernetes.io/projected/cade7b49-ca45-490c-870d-9c9b9eea59cd-kube-api-access-xk9l8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.371381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.371342    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cade7b49-ca45-490c-870d-9c9b9eea59cd-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.371615 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.371602    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cade7b49-ca45-490c-870d-9c9b9eea59cd-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.381368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.381340    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk9l8\" (UniqueName: \"kubernetes.io/projected/cade7b49-ca45-490c-870d-9c9b9eea59cd-kube-api-access-xk9l8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-jch2v\" (UID: \"cade7b49-ca45-490c-870d-9c9b9eea59cd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.543089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.543042    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"
Oct 30 19:22:20.825127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:20.825100    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v"]
Oct 30 19:22:20.826375 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:22:20.826350    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcade7b49_ca45_490c_870d_9c9b9eea59cd.slice/crio-da803286bf5f744d62947f5044fa0457c5e4aa6ece419b86dca32ad6afeacc1a WatchSource:0}: Error finding container da803286bf5f744d62947f5044fa0457c5e4aa6ece419b86dca32ad6afeacc1a: Status 404 returned error can't find the container with id da803286bf5f744d62947f5044fa0457c5e4aa6ece419b86dca32ad6afeacc1a
Oct 30 19:22:21.252818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:21.252779    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v" event={"ID":"cade7b49-ca45-490c-870d-9c9b9eea59cd","Type":"ContainerStarted","Data":"da803286bf5f744d62947f5044fa0457c5e4aa6ece419b86dca32ad6afeacc1a"}
Oct 30 19:22:24.273973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:24.273934    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v" event={"ID":"cade7b49-ca45-490c-870d-9c9b9eea59cd","Type":"ContainerStarted","Data":"c9749541139ab8102c81b9ccb6732c0eacf8fa7dab4432a5a644d210049ec057"}
Oct 30 19:22:24.300587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:24.300542    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-jch2v" podStartSLOduration=1.552963538 podStartE2EDuration="4.300530744s" totalImagesPullingTime="2.747567206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:24.299543801 +0000 UTC m=+209.269184637" watchObservedRunningTime="2026-10-30 19:22:24.300530744 +0000 UTC m=+209.270171582"
Oct 30 19:22:27.440590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.440556    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"]
Oct 30 19:22:27.442941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.442925    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:27.450054 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.450031    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kk4m5\""
Oct 30 19:22:27.450054 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.450037    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 19:22:27.450618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.450597    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 19:22:27.450740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.450665    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 19:22:27.484088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.484064    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"]
Oct 30 19:22:27.526750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.526724    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jpn\" (UniqueName: \"kubernetes.io/projected/ab0b4b4e-4979-40ac-9be8-3864e5ab9628-kube-api-access-q5jpn\") pod \"dns-operator-controller-manager-8557cbdb84-sr2jh\" (UID: \"ab0b4b4e-4979-40ac-9be8-3864e5ab9628\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:27.627847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.627815    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jpn\" (UniqueName: \"kubernetes.io/projected/ab0b4b4e-4979-40ac-9be8-3864e5ab9628-kube-api-access-q5jpn\") pod \"dns-operator-controller-manager-8557cbdb84-sr2jh\" (UID: \"ab0b4b4e-4979-40ac-9be8-3864e5ab9628\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:27.638979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.638951    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jpn\" (UniqueName: \"kubernetes.io/projected/ab0b4b4e-4979-40ac-9be8-3864e5ab9628-kube-api-access-q5jpn\") pod \"dns-operator-controller-manager-8557cbdb84-sr2jh\" (UID: \"ab0b4b4e-4979-40ac-9be8-3864e5ab9628\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:27.754841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:27.754809    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:28.032634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:28.032611    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"]
Oct 30 19:22:28.035185 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:22:28.035153    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0b4b4e_4979_40ac_9be8_3864e5ab9628.slice/crio-50f938775a1b336eadf02bb920d024b51ee4ba4b1942489b5de5062a91bf443c WatchSource:0}: Error finding container 50f938775a1b336eadf02bb920d024b51ee4ba4b1942489b5de5062a91bf443c: Status 404 returned error can't find the container with id 50f938775a1b336eadf02bb920d024b51ee4ba4b1942489b5de5062a91bf443c
Oct 30 19:22:28.292746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:28.292665    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh" event={"ID":"ab0b4b4e-4979-40ac-9be8-3864e5ab9628","Type":"ContainerStarted","Data":"50f938775a1b336eadf02bb920d024b51ee4ba4b1942489b5de5062a91bf443c"}
Oct 30 19:22:29.245364 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:29.245327    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-668f68f55d-sbzvc"
Oct 30 19:22:31.310698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:31.310645    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh" event={"ID":"ab0b4b4e-4979-40ac-9be8-3864e5ab9628","Type":"ContainerStarted","Data":"314c311232f9a92f6bdb44de723ef3e13d744385ac5b3431f09889027fea12dd"}
Oct 30 19:22:31.311057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:31.310730    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:31.338292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:31.338237    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh" podStartSLOduration=1.5738783760000001 podStartE2EDuration="4.338218823s" totalImagesPullingTime="2.764340447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:31.33662686 +0000 UTC m=+216.306267695" watchObservedRunningTime="2026-10-30 19:22:31.338218823 +0000 UTC m=+216.307859660"
Oct 30 19:22:32.272817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.272785    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"]
Oct 30 19:22:32.279037 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.276722    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:32.290369 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.290344    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vb92v\""
Oct 30 19:22:32.292229 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.292206    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"]
Oct 30 19:22:32.367857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.367826    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47stf\" (UniqueName: \"kubernetes.io/projected/13ae1ec1-cef4-4a7a-aadf-4197b010f02e-kube-api-access-47stf\") pod \"authorino-operator-7b7bb4bc77-sxzhz\" (UID: \"13ae1ec1-cef4-4a7a-aadf-4197b010f02e\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:32.468871 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.468841    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47stf\" (UniqueName: \"kubernetes.io/projected/13ae1ec1-cef4-4a7a-aadf-4197b010f02e-kube-api-access-47stf\") pod \"authorino-operator-7b7bb4bc77-sxzhz\" (UID: \"13ae1ec1-cef4-4a7a-aadf-4197b010f02e\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:32.484541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.484512    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47stf\" (UniqueName: \"kubernetes.io/projected/13ae1ec1-cef4-4a7a-aadf-4197b010f02e-kube-api-access-47stf\") pod \"authorino-operator-7b7bb4bc77-sxzhz\" (UID: \"13ae1ec1-cef4-4a7a-aadf-4197b010f02e\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:32.590232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.590182    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:32.861199 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:32.861174    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"]
Oct 30 19:22:32.863164 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:22:32.863139    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ae1ec1_cef4_4a7a_aadf_4197b010f02e.slice/crio-ac8471c88e3e103b5eff31caa5d9e8765dd3f43708cbb133c27d3ef3fd565f01 WatchSource:0}: Error finding container ac8471c88e3e103b5eff31caa5d9e8765dd3f43708cbb133c27d3ef3fd565f01: Status 404 returned error can't find the container with id ac8471c88e3e103b5eff31caa5d9e8765dd3f43708cbb133c27d3ef3fd565f01
Oct 30 19:22:33.322020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:33.321984    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz" event={"ID":"13ae1ec1-cef4-4a7a-aadf-4197b010f02e","Type":"ContainerStarted","Data":"ac8471c88e3e103b5eff31caa5d9e8765dd3f43708cbb133c27d3ef3fd565f01"}
Oct 30 19:22:34.985372 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:22:34.985335    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:22:35.332439 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:35.332344    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz" event={"ID":"13ae1ec1-cef4-4a7a-aadf-4197b010f02e","Type":"ContainerStarted","Data":"418855064c92e99033beaaa5dd774799ab67abef01fb59dde1fc6e74da8a8958"}
Oct 30 19:22:35.332589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:35.332512    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:35.357519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:35.357473    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz" podStartSLOduration=1.685124589 podStartE2EDuration="3.357463253s" totalImagesPullingTime="1.672338664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:35.356027817 +0000 UTC m=+220.325668643" watchObservedRunningTime="2026-10-30 19:22:35.357463253 +0000 UTC m=+220.327104087"
Oct 30 19:22:42.317031 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:42.316999    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-sr2jh"
Oct 30 19:22:43.712017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.711985    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:22:43.715408 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.715391    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.720960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.720935    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vlzxc\""
Oct 30 19:22:43.727125 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.727103    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:22:43.757567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.757540    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.757671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.757569    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l74rg\" (UniqueName: \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.858911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.858883    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.858911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.858914    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l74rg\" (UniqueName: \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.859319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.859298    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:43.868736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:43.868714    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l74rg\" (UniqueName: \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\") pod \"kuadrant-operator-controller-manager-678d87766f-pjpn2\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:44.029141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:44.029087    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:44.314343 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:44.314321    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:22:44.316136 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:22:44.316106    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a72a87_9a52_4842_8c20_257e72176f15.slice/crio-9ca76f980e1d4c8e916bdba9028ee9208009159a34d02b9459829e744d9f2c3f WatchSource:0}: Error finding container 9ca76f980e1d4c8e916bdba9028ee9208009159a34d02b9459829e744d9f2c3f: Status 404 returned error can't find the container with id 9ca76f980e1d4c8e916bdba9028ee9208009159a34d02b9459829e744d9f2c3f
Oct 30 19:22:44.372958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:44.372926    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" event={"ID":"b6a72a87-9a52-4842-8c20-257e72176f15","Type":"ContainerStarted","Data":"9ca76f980e1d4c8e916bdba9028ee9208009159a34d02b9459829e744d9f2c3f"}
Oct 30 19:22:46.340558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:46.340526    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-sxzhz"
Oct 30 19:22:47.152011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.151977    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"]
Oct 30 19:22:47.156321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.156300    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.172901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.172876    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gswhc\""
Oct 30 19:22:47.174819 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.174792    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"]
Oct 30 19:22:47.286471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286439    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286497    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286520    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286549    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhhfd\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-kube-api-access-zhhfd\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286622    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286741    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286787    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286826    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.286972 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.286952    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388208    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388251    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388274    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388309    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388331    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388358    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhhfd\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-kube-api-access-zhhfd\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388396    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388431    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388483    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388659    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.388745 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388669    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.389205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.388933    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.389269 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.389212    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.389327 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.389297    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.391020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.391002    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.398288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.398206    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhhfd\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-kube-api-access-zhhfd\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.398485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.398464    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.399158 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.399124    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aba00d88-8a0d-48ce-a232-ee2521c70bb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b\" (UID: \"aba00d88-8a0d-48ce-a232-ee2521c70bb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.475550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.475523    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:47.763870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.763847    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"]
Oct 30 19:22:47.773703 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:22:47.765882    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba00d88_8a0d_48ce_a232_ee2521c70bb9.slice/crio-56322e6502c6ceefd958cc19fba90927ca5712c428eb78d06f10654afa83a404 WatchSource:0}: Error finding container 56322e6502c6ceefd958cc19fba90927ca5712c428eb78d06f10654afa83a404: Status 404 returned error can't find the container with id 56322e6502c6ceefd958cc19fba90927ca5712c428eb78d06f10654afa83a404
Oct 30 19:22:47.773703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.768251    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:47.773703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.768312    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:47.773703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:47.768352    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:48.394662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:48.394625    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b" event={"ID":"aba00d88-8a0d-48ce-a232-ee2521c70bb9","Type":"ContainerStarted","Data":"d4e88add414fe01016bfb24fb8e72d775612a79f7d27e187f00b5da3d521f34c"}
Oct 30 19:22:48.395116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:48.394708    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b" event={"ID":"aba00d88-8a0d-48ce-a232-ee2521c70bb9","Type":"ContainerStarted","Data":"56322e6502c6ceefd958cc19fba90927ca5712c428eb78d06f10654afa83a404"}
Oct 30 19:22:48.475735 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:48.475710    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:48.481205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:48.481179    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:48.506869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:48.506813    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b" podStartSLOduration=1.506796078 podStartE2EDuration="1.506796078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:48.419007156 +0000 UTC m=+233.388647992" watchObservedRunningTime="2026-10-30 19:22:48.506796078 +0000 UTC m=+233.476436916"
Oct 30 19:22:49.401099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:49.401059    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" event={"ID":"b6a72a87-9a52-4842-8c20-257e72176f15","Type":"ContainerStarted","Data":"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"}
Oct 30 19:22:49.401508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:49.401413    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:22:49.401508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:49.401456    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:49.402353 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:49.402336    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b"
Oct 30 19:22:49.424375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:22:49.424333    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" podStartSLOduration=2.118482876 podStartE2EDuration="6.424293271s" totalImagesPullingTime="4.305810395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:49.423780984 +0000 UTC m=+234.393421820" watchObservedRunningTime="2026-10-30 19:22:49.424293271 +0000 UTC m=+234.393934106"
Oct 30 19:23:00.407552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:00.407526    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:23:02.081407 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.081361    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:02.086978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.086955    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.093885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.093837    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5ptsd\""
Oct 30 19:23:02.107522 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.107496    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:02.143370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.143349    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 19:23:02.152908 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.152884    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:02.204886 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.204862    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.204979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.204895    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb62k\" (UniqueName: \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.306140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.306113    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.306263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.306152    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb62k\" (UniqueName: \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.306792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.306773    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.319440 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.319419    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb62k\" (UniqueName: \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\") pod \"limitador-limitador-74dfcfb894-82q94\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.401335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.401280    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:02.709954 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:02.709929    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:02.712664 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:02.712632    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03bd070_91ee_4289_9d84_662312faa7a7.slice/crio-779e8d4b78da3e530564192148e555f5365cb6f2b7b7049318c673809b09d2e2 WatchSource:0}: Error finding container 779e8d4b78da3e530564192148e555f5365cb6f2b7b7049318c673809b09d2e2: Status 404 returned error can't find the container with id 779e8d4b78da3e530564192148e555f5365cb6f2b7b7049318c673809b09d2e2
Oct 30 19:23:03.037765 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.037696    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:03.040977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.040959    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:03.046408 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.046379    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-plmdz\""
Oct 30 19:23:03.055103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.055082    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:03.113548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.113519    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9gdv\" (UniqueName: \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\") pod \"authorino-57f7f8659f-4vkfx\" (UID: \"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203\") " pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:03.214527 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.214501    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gdv\" (UniqueName: \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\") pod \"authorino-57f7f8659f-4vkfx\" (UID: \"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203\") " pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:03.227518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.227491    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9gdv\" (UniqueName: \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\") pod \"authorino-57f7f8659f-4vkfx\" (UID: \"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203\") " pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:03.248293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.248271    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:23:03.250551 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.250537    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:23:03.264687 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.264654    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:23:03.315020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.314965    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhbc\" (UniqueName: \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\") pod \"authorino-56c6c9ddd5-pksxs\" (UID: \"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e\") " pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:23:03.370209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.370187    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:03.416636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.416606    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhbc\" (UniqueName: \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\") pod \"authorino-56c6c9ddd5-pksxs\" (UID: \"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e\") " pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:23:03.430732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.430667    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhbc\" (UniqueName: \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\") pod \"authorino-56c6c9ddd5-pksxs\" (UID: \"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e\") " pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:23:03.468758 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.468733    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" event={"ID":"f03bd070-91ee-4289-9d84-662312faa7a7","Type":"ContainerStarted","Data":"779e8d4b78da3e530564192148e555f5365cb6f2b7b7049318c673809b09d2e2"}
Oct 30 19:23:03.561231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.561210    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:23:03.643072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.643047    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:03.644607 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:03.644580    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e3e1cc5_a75c_4f6e_88aa_45e171a0b203.slice/crio-35b871b5efcc93ce3d46bbc082f38ed77dce1df4b15594aea14237d7fc865314 WatchSource:0}: Error finding container 35b871b5efcc93ce3d46bbc082f38ed77dce1df4b15594aea14237d7fc865314: Status 404 returned error can't find the container with id 35b871b5efcc93ce3d46bbc082f38ed77dce1df4b15594aea14237d7fc865314
Oct 30 19:23:03.834183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:03.834160    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:23:03.835640 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:03.835611    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod849b2d3e_c18f_49c5_b4f4_35c6f5c3454e.slice/crio-fba4301c4bfd86c0947b60027eccd629192466655bf49ed6b4df7f8fe82cf1f8 WatchSource:0}: Error finding container fba4301c4bfd86c0947b60027eccd629192466655bf49ed6b4df7f8fe82cf1f8: Status 404 returned error can't find the container with id fba4301c4bfd86c0947b60027eccd629192466655bf49ed6b4df7f8fe82cf1f8
Oct 30 19:23:04.474650 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:04.474619    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" event={"ID":"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e","Type":"ContainerStarted","Data":"fba4301c4bfd86c0947b60027eccd629192466655bf49ed6b4df7f8fe82cf1f8"}
Oct 30 19:23:04.476062 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:04.476037    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" event={"ID":"f03bd070-91ee-4289-9d84-662312faa7a7","Type":"ContainerStarted","Data":"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"}
Oct 30 19:23:04.476184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:04.476098    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:04.477215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:04.477193    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" event={"ID":"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203","Type":"ContainerStarted","Data":"35b871b5efcc93ce3d46bbc082f38ed77dce1df4b15594aea14237d7fc865314"}
Oct 30 19:23:04.494107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:04.494066    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" podStartSLOduration=1.065494434 podStartE2EDuration="2.494052862s" totalImagesPullingTime="1.428558428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:04.493032521 +0000 UTC m=+249.462673357" watchObservedRunningTime="2026-10-30 19:23:04.494052862 +0000 UTC m=+249.463693696"
Oct 30 19:23:04.992965 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:23:04.992928    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:23:06.489250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:06.489207    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" event={"ID":"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e","Type":"ContainerStarted","Data":"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"}
Oct 30 19:23:06.492064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:06.492031    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" event={"ID":"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203","Type":"ContainerStarted","Data":"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"}
Oct 30 19:23:06.506016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:06.505973    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" podStartSLOduration=0.946513292 podStartE2EDuration="3.505955852s" totalImagesPullingTime="2.55944256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:06.505492041 +0000 UTC m=+251.475132877" watchObservedRunningTime="2026-10-30 19:23:06.505955852 +0000 UTC m=+251.475596689"
Oct 30 19:23:06.523341 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:06.522602    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" podStartSLOduration=0.778755621 podStartE2EDuration="3.522586362s" totalImagesPullingTime="2.743830741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:06.520822089 +0000 UTC m=+251.490462928" watchObservedRunningTime="2026-10-30 19:23:06.522586362 +0000 UTC m=+251.492227198"
Oct 30 19:23:06.552034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:06.552000    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:08.501860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:08.501826    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" podUID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" containerName="authorino" containerID="cri-o://c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766" gracePeriod=30
Oct 30 19:23:08.752556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:08.752503    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:08.864417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:08.864389    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\" (UniqueName: \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\") pod \"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203\" (UID: \"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203\") "
Oct 30 19:23:08.866566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:08.866545    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv" pod "1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" (UID: "1e3e1cc5-a75c-4f6e-88aa-45e171a0b203"). InnerVolumeSpecName "kube-api-access-b9gdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:08.965118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:08.965090    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9gdv\" (UniqueName: \"kubernetes.io/projected/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203-kube-api-access-b9gdv\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:09.507576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.507547    5033 generic.go:356] "Generic (PLEG): container finished" podID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" containerID="c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766" exitCode=0
Oct 30 19:23:09.507936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.507591    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-4vkfx"
Oct 30 19:23:09.507936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.507618    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" event={"ID":"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203","Type":"ContainerDied","Data":"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"}
Oct 30 19:23:09.507936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.507651    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-4vkfx" event={"ID":"1e3e1cc5-a75c-4f6e-88aa-45e171a0b203","Type":"ContainerDied","Data":"35b871b5efcc93ce3d46bbc082f38ed77dce1df4b15594aea14237d7fc865314"}
Oct 30 19:23:09.507936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.507695    5033 scope.go:122] "RemoveContainer" containerID="c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"
Oct 30 19:23:09.516004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.515986    5033 scope.go:122] "RemoveContainer" containerID="c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"
Oct 30 19:23:09.516232 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:23:09.516214    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766\": container with ID starting with c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766 not found: ID does not exist" containerID="c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"
Oct 30 19:23:09.516287 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.516243    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766"} err="failed to get container status \"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766\": rpc error: code = NotFound desc = could not find container \"c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766\": container with ID starting with c2bc683f0e5fb2dfa1012975c28fbeb7041830a92f4a5fec23ebce54b0bff766 not found: ID does not exist"
Oct 30 19:23:09.527444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.527425    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:09.529611 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:09.529590    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-4vkfx"]
Oct 30 19:23:11.167295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:11.167264    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" path="/var/lib/kubelet/pods/1e3e1cc5-a75c-4f6e-88aa-45e171a0b203/volumes"
Oct 30 19:23:15.409079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:15.409046    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:15.411520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:15.409243    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" podUID="f03bd070-91ee-4289-9d84-662312faa7a7" containerName="limitador" containerID="cri-o://3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177" gracePeriod=30
Oct 30 19:23:15.411520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:15.409905    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:15.950228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:15.950208    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:16.113617 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.113552    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\" (UniqueName: \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\") pod \"f03bd070-91ee-4289-9d84-662312faa7a7\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") "
Oct 30 19:23:16.113617 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.113599    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\" (UniqueName: \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\") pod \"f03bd070-91ee-4289-9d84-662312faa7a7\" (UID: \"f03bd070-91ee-4289-9d84-662312faa7a7\") "
Oct 30 19:23:16.113994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.113974    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file" pod "f03bd070-91ee-4289-9d84-662312faa7a7" (UID: "f03bd070-91ee-4289-9d84-662312faa7a7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:16.115531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.115513    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k" pod "f03bd070-91ee-4289-9d84-662312faa7a7" (UID: "f03bd070-91ee-4289-9d84-662312faa7a7"). InnerVolumeSpecName "kube-api-access-rb62k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:16.214985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.214952    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb62k\" (UniqueName: \"kubernetes.io/projected/f03bd070-91ee-4289-9d84-662312faa7a7-kube-api-access-rb62k\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:16.214985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.214989    5033 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f03bd070-91ee-4289-9d84-662312faa7a7-config-file\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:16.540815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.540785    5033 generic.go:356] "Generic (PLEG): container finished" podID="f03bd070-91ee-4289-9d84-662312faa7a7" containerID="3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177" exitCode=0
Oct 30 19:23:16.541215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.540826    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94"
Oct 30 19:23:16.541215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.540874    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" event={"ID":"f03bd070-91ee-4289-9d84-662312faa7a7","Type":"ContainerDied","Data":"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"}
Oct 30 19:23:16.541215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.540920    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-82q94" event={"ID":"f03bd070-91ee-4289-9d84-662312faa7a7","Type":"ContainerDied","Data":"779e8d4b78da3e530564192148e555f5365cb6f2b7b7049318c673809b09d2e2"}
Oct 30 19:23:16.541215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.540943    5033 scope.go:122] "RemoveContainer" containerID="3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"
Oct 30 19:23:16.549522 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.549502    5033 scope.go:122] "RemoveContainer" containerID="3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"
Oct 30 19:23:16.549809 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:23:16.549791    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177\": container with ID starting with 3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177 not found: ID does not exist" containerID="3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"
Oct 30 19:23:16.549904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.549821    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177"} err="failed to get container status \"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177\": rpc error: code = NotFound desc = could not find container \"3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177\": container with ID starting with 3bebee06bba55f39a6eda5af1c3710345a6333051dd74497cc4e8fa3fc35a177 not found: ID does not exist"
Oct 30 19:23:16.565919 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.565896    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:16.571105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:16.571085    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-82q94"]
Oct 30 19:23:17.166923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:17.166887    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f03bd070-91ee-4289-9d84-662312faa7a7" path="/var/lib/kubelet/pods/f03bd070-91ee-4289-9d84-662312faa7a7/volumes"
Oct 30 19:23:20.363526 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363489    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"]
Oct 30 19:23:20.363969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363902    5033 cpu_manager.go:414] "containerMap: removing container" podUID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" containerName="authorino"
Oct 30 19:23:20.363969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363916    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" containerName="authorino"
Oct 30 19:23:20.363969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363936    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f03bd070-91ee-4289-9d84-662312faa7a7" containerName="limitador"
Oct 30 19:23:20.363969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363941    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f03bd070-91ee-4289-9d84-662312faa7a7" containerName="limitador"
Oct 30 19:23:20.364122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.363998    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="f03bd070-91ee-4289-9d84-662312faa7a7" containerName="limitador"
Oct 30 19:23:20.364122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.364009    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e3e1cc5-a75c-4f6e-88aa-45e171a0b203" containerName="authorino"
Oct 30 19:23:20.366712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.366697    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.374282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.374254    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 19:23:20.376036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.376013    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:23:20.376036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.376025    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:23:20.376216 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.376089    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ccnlz\""
Oct 30 19:23:20.376700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.376658    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"]
Oct 30 19:23:20.545247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.545219    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ec0512be-25bd-486b-9310-54f908948157-data\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.545390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.545330    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk7zg\" (UniqueName: \"kubernetes.io/projected/ec0512be-25bd-486b-9310-54f908948157-kube-api-access-bk7zg\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.646144 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.646077    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk7zg\" (UniqueName: \"kubernetes.io/projected/ec0512be-25bd-486b-9310-54f908948157-kube-api-access-bk7zg\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.646144 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.646115    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ec0512be-25bd-486b-9310-54f908948157-data\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.646420 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.646406    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ec0512be-25bd-486b-9310-54f908948157-data\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.656250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.656224    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk7zg\" (UniqueName: \"kubernetes.io/projected/ec0512be-25bd-486b-9310-54f908948157-kube-api-access-bk7zg\") pod \"postgres-7cdbf6dc44-xddcz\" (UID: \"ec0512be-25bd-486b-9310-54f908948157\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.679972 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.679949    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:20.951712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:20.951666    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"]
Oct 30 19:23:20.953537 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:20.953505    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0512be_25bd_486b_9310_54f908948157.slice/crio-ead8766a4b94ee5b5b5ee23bc498e38f7b0f78236a2f4ce05992c175fc203f12 WatchSource:0}: Error finding container ead8766a4b94ee5b5b5ee23bc498e38f7b0f78236a2f4ce05992c175fc203f12: Status 404 returned error can't find the container with id ead8766a4b94ee5b5b5ee23bc498e38f7b0f78236a2f4ce05992c175fc203f12
Oct 30 19:23:21.566593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:21.566556    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz" event={"ID":"ec0512be-25bd-486b-9310-54f908948157","Type":"ContainerStarted","Data":"ead8766a4b94ee5b5b5ee23bc498e38f7b0f78236a2f4ce05992c175fc203f12"}
Oct 30 19:23:26.599773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:26.599733    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz" event={"ID":"ec0512be-25bd-486b-9310-54f908948157","Type":"ContainerStarted","Data":"f07539570ad0bdbea1b18c6d1ba3e4a9a9fdff7c5f4c24ab0f0892fc5770e578"}
Oct 30 19:23:26.600118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:26.599861    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:26.638401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:26.638301    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz" podStartSLOduration=1.212129787 podStartE2EDuration="6.638290151s" totalImagesPullingTime="5.426160364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:26.638042408 +0000 UTC m=+271.607683245" watchObservedRunningTime="2026-10-30 19:23:26.638290151 +0000 UTC m=+271.607930983"
Oct 30 19:23:32.632802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:32.632776    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xddcz"
Oct 30 19:23:34.984922 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:23:34.984889    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:23:41.556582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.556549    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:23:41.560628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.560612    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.568952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.568905    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:23:41.568952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.568935    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:23:41.570649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.570629    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l2kjc\""
Oct 30 19:23:41.573603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.573578    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 19:23:41.573781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.573597    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:23:41.573781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.573619    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 19:23:41.609637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.609604    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7dl\" (UniqueName: \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.609917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.609787    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.609917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.609852    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.652453 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.652423    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:23:41.710764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.710737    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7dl\" (UniqueName: \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.710910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.710773    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.710910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.710792    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.713253 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.713223    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.713354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.713340    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.744646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.744625    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7dl\" (UniqueName: \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\") pod \"maas-controller-579cf7c6d-qk67h\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") " pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.874837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.874756    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:41.933747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.933637    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:23:41.940812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.939794    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:41.960721 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:41.960693    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:23:42.013927 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.013902    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.014043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.013945    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pws7h\" (UniqueName: \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.014043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.013966    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.115251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.115204    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.115366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.115305    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.115366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.115360    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws7h\" (UniqueName: \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.117597 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.117575    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.117702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.117621    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.151571 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.151520    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pws7h\" (UniqueName: \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\") pod \"maas-controller-78455b774c-skgjw\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") " pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.169574 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.169552    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:23:42.173710 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:42.173668    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b0ab9b_5f7b_4b0a_8fcd_eaf00109d8f9.slice/crio-c2d65cc35f8f252b9ec2e2f334e04ef7132f61a521189139e3ec7ae5e849a6af WatchSource:0}: Error finding container c2d65cc35f8f252b9ec2e2f334e04ef7132f61a521189139e3ec7ae5e849a6af: Status 404 returned error can't find the container with id c2d65cc35f8f252b9ec2e2f334e04ef7132f61a521189139e3ec7ae5e849a6af
Oct 30 19:23:42.256388 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.256357    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:42.486428 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.486397    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-pwkmf"]
Oct 30 19:23:42.490585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.490568    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.510875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.510854    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 19:23:42.511169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.511146    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qmrl6\""
Oct 30 19:23:42.513937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.513914    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-pwkmf"]
Oct 30 19:23:42.521716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.521695    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/891bf17a-d1a7-41dc-b318-df4c72b3ec05-cert\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.521813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.521773    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcj5p\" (UniqueName: \"kubernetes.io/projected/891bf17a-d1a7-41dc-b318-df4c72b3ec05-kube-api-access-lcj5p\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.530958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.530935    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:23:42.534803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.534785    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.566791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.566759    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:23:42.569005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.568977    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:23:42.571313 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:42.571293    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7827f4c0_6c4d_48e4_959a_47188b2693ab.slice/crio-d18002924d393fb5ca2a082f5f42790f85b91cc00aa5c6ff16505c1a5168c9c5 WatchSource:0}: Error finding container d18002924d393fb5ca2a082f5f42790f85b91cc00aa5c6ff16505c1a5168c9c5: Status 404 returned error can't find the container with id d18002924d393fb5ca2a082f5f42790f85b91cc00aa5c6ff16505c1a5168c9c5
Oct 30 19:23:42.608598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.608575    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kwnzx\""
Oct 30 19:23:42.608740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.608619    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 19:23:42.623185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.623161    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcj5p\" (UniqueName: \"kubernetes.io/projected/891bf17a-d1a7-41dc-b318-df4c72b3ec05-kube-api-access-lcj5p\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.623312 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.623200    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6fg6\" (UniqueName: \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.623312 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.623224    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.623440 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.623387    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/891bf17a-d1a7-41dc-b318-df4c72b3ec05-cert\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.625786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.625765    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/891bf17a-d1a7-41dc-b318-df4c72b3ec05-cert\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.635442 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.635418    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcj5p\" (UniqueName: \"kubernetes.io/projected/891bf17a-d1a7-41dc-b318-df4c72b3ec05-kube-api-access-lcj5p\") pod \"kserve-controller-manager-776cc66d47-pwkmf\" (UID: \"891bf17a-d1a7-41dc-b318-df4c72b3ec05\") " pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.683149 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.683110    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-qk67h" event={"ID":"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9","Type":"ContainerStarted","Data":"c2d65cc35f8f252b9ec2e2f334e04ef7132f61a521189139e3ec7ae5e849a6af"}
Oct 30 19:23:42.684454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.684429    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78455b774c-skgjw" event={"ID":"7827f4c0-6c4d-48e4-959a-47188b2693ab","Type":"ContainerStarted","Data":"d18002924d393fb5ca2a082f5f42790f85b91cc00aa5c6ff16505c1a5168c9c5"}
Oct 30 19:23:42.724575 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.724553    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6fg6\" (UniqueName: \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.724665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.724582    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.727112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.727092    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.737968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.737916    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6fg6\" (UniqueName: \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\") pod \"llmisvc-controller-manager-869d5dcc5d-4xsh2\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:42.803768 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.803744    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:42.848202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:42.848173    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:43.105916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:43.105886    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-pwkmf"]
Oct 30 19:23:43.108132 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:43.108100    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891bf17a_d1a7_41dc_b318_df4c72b3ec05.slice/crio-bba15e0bf2789920b5561396795c614c3bf34970e5787119f8d4fd6b7b1b9b0b WatchSource:0}: Error finding container bba15e0bf2789920b5561396795c614c3bf34970e5787119f8d4fd6b7b1b9b0b: Status 404 returned error can't find the container with id bba15e0bf2789920b5561396795c614c3bf34970e5787119f8d4fd6b7b1b9b0b
Oct 30 19:23:43.148901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:43.148860    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:23:43.151799 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:23:43.151759    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod901f3e83_8736_4eb5_91c8_c7e4f9ad3bd8.slice/crio-79314041e3790f3a7546ad9c3601f6af9657a09fb2b31a132c9b19c2ab42a537 WatchSource:0}: Error finding container 79314041e3790f3a7546ad9c3601f6af9657a09fb2b31a132c9b19c2ab42a537: Status 404 returned error can't find the container with id 79314041e3790f3a7546ad9c3601f6af9657a09fb2b31a132c9b19c2ab42a537
Oct 30 19:23:43.727071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:43.726785    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" event={"ID":"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8","Type":"ContainerStarted","Data":"79314041e3790f3a7546ad9c3601f6af9657a09fb2b31a132c9b19c2ab42a537"}
Oct 30 19:23:43.735282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:43.735202    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf" event={"ID":"891bf17a-d1a7-41dc-b318-df4c72b3ec05","Type":"ContainerStarted","Data":"bba15e0bf2789920b5561396795c614c3bf34970e5787119f8d4fd6b7b1b9b0b"}
Oct 30 19:23:46.769121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:46.768802    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-qk67h" event={"ID":"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9","Type":"ContainerStarted","Data":"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"}
Oct 30 19:23:46.770251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:46.770230    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:46.816901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:46.816849    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-qk67h" podStartSLOduration=1.691577159 podStartE2EDuration="5.816832765s" totalImagesPullingTime="4.125255606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:46.815496582 +0000 UTC m=+291.785137441" watchObservedRunningTime="2026-10-30 19:23:46.816832765 +0000 UTC m=+291.786473601"
Oct 30 19:23:47.777875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.777606    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" event={"ID":"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8","Type":"ContainerStarted","Data":"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"}
Oct 30 19:23:47.777875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.777731    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:23:47.779740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.779711    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78455b774c-skgjw" event={"ID":"7827f4c0-6c4d-48e4-959a-47188b2693ab","Type":"ContainerStarted","Data":"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"}
Oct 30 19:23:47.779892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.779782    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:47.781389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.781366    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf" event={"ID":"891bf17a-d1a7-41dc-b318-df4c72b3ec05","Type":"ContainerStarted","Data":"faff8c27424e1794dace17645c4b4b9545f4e9a89b31750b43898692a0028c53"}
Oct 30 19:23:47.781584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.781505    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:23:47.798189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.798118    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" podStartSLOduration=1.8346313159999998 podStartE2EDuration="5.798107345s" totalImagesPullingTime="3.963476029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:47.796895257 +0000 UTC m=+292.766536093" watchObservedRunningTime="2026-10-30 19:23:47.798107345 +0000 UTC m=+292.767748180"
Oct 30 19:23:47.817826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.817789    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-78455b774c-skgjw" podStartSLOduration=2.502679233 podStartE2EDuration="6.817778557s" totalImagesPullingTime="4.315099324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:47.81635327 +0000 UTC m=+292.785994106" watchObservedRunningTime="2026-10-30 19:23:47.817778557 +0000 UTC m=+292.787419392"
Oct 30 19:23:47.836948 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:47.836907    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf" podStartSLOduration=1.618445848 podStartE2EDuration="5.836896013s" totalImagesPullingTime="4.218450165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:47.835163861 +0000 UTC m=+292.804804696" watchObservedRunningTime="2026-10-30 19:23:47.836896013 +0000 UTC m=+292.806536849"
Oct 30 19:23:55.199912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:55.199884    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:23:55.200257 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:55.200013    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:23:58.791254 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:58.791228    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:58.791717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:58.791292    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:23:58.870562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:58.870522    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:23:58.870746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:58.870718    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-qk67h" podUID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" containerName="manager" containerID="cri-o://6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537" gracePeriod=30
Oct 30 19:23:59.112510 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.112487    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:59.197396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.197369    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\" (UniqueName: \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\") pod \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") "
Oct 30 19:23:59.197513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.197437    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\") pod \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") "
Oct 30 19:23:59.197576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.197526    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\") pod \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\" (UID: \"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9\") "
Oct 30 19:23:59.199548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.199522    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs" pod "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" (UID: "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:59.199685 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.199624    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert" pod "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" (UID: "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:59.199685 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.199651    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl" pod "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" (UID: "e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9"). InnerVolumeSpecName "kube-api-access-hn7dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:59.298631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.298611    5033 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-metrics-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:59.298631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.298632    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn7dl\" (UniqueName: \"kubernetes.io/projected/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-kube-api-access-hn7dl\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:59.298794 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.298642    5033 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9-webhook-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:59.852986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.852954    5033 generic.go:356] "Generic (PLEG): container finished" podID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" containerID="6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537" exitCode=0
Oct 30 19:23:59.853359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.853009    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-qk67h"
Oct 30 19:23:59.853359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.853049    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-qk67h" event={"ID":"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9","Type":"ContainerDied","Data":"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"}
Oct 30 19:23:59.853359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.853114    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-qk67h" event={"ID":"e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9","Type":"ContainerDied","Data":"c2d65cc35f8f252b9ec2e2f334e04ef7132f61a521189139e3ec7ae5e849a6af"}
Oct 30 19:23:59.853359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.853147    5033 scope.go:122] "RemoveContainer" containerID="6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"
Oct 30 19:23:59.862612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.862590    5033 scope.go:122] "RemoveContainer" containerID="6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"
Oct 30 19:23:59.862902 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:23:59.862884    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537\": container with ID starting with 6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537 not found: ID does not exist" containerID="6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"
Oct 30 19:23:59.862971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.862909    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537"} err="failed to get container status \"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537\": rpc error: code = NotFound desc = could not find container \"6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537\": container with ID starting with 6355f2b1f3f8db37bbc84ebd0316b820e4c910d1bdc2ba9c995e7e7c9d145537 not found: ID does not exist"
Oct 30 19:23:59.876959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.876938    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:23:59.881471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:23:59.881451    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-qk67h"]
Oct 30 19:24:01.168269 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:01.168241    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" path="/var/lib/kubelet/pods/e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9/volumes"
Oct 30 19:24:04.985371 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:24:04.985338    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:24:18.792520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:18.792427    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:24:18.794990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:18.794967    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-pwkmf"
Oct 30 19:24:28.294858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.294819    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:28.295408 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.295385    5033 cpu_manager.go:414] "containerMap: removing container" podUID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" containerName="manager"
Oct 30 19:24:28.295494 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.295411    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" containerName="manager"
Oct 30 19:24:28.295550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.295529    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9b0ab9b-5f7b-4b0a-8fcd-eaf00109d8f9" containerName="manager"
Oct 30 19:24:28.297714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.297667    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.308119 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.308096    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:28.325048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.325028    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 19:24:28.424107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.424081    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8l7\" (UniqueName: \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.424261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.424241    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.525366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.525342    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.525489 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.525387    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8l7\" (UniqueName: \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.527787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.527762    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.537327 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.537305    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8l7\" (UniqueName: \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\") pod \"authorino-6889cbd8c4-bklgv\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") " pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.590004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.589940    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:28.590126 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.590114    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:28.874986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.874921    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:28.878053 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:24:28.878024    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ac1ea27_5ca6_48fe_ae69_f51e26d49cc4.slice/crio-2db4fa923bca1fd8160b5e8cc4aea1a80d949311c7fa5a5c50a5946e31610c8c WatchSource:0}: Error finding container 2db4fa923bca1fd8160b5e8cc4aea1a80d949311c7fa5a5c50a5946e31610c8c: Status 404 returned error can't find the container with id 2db4fa923bca1fd8160b5e8cc4aea1a80d949311c7fa5a5c50a5946e31610c8c
Oct 30 19:24:28.879423 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.879404    5033 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:24:28.995026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:28.994987    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" event={"ID":"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4","Type":"ContainerStarted","Data":"2db4fa923bca1fd8160b5e8cc4aea1a80d949311c7fa5a5c50a5946e31610c8c"}
Oct 30 19:24:30.004537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.004501    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" event={"ID":"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4","Type":"ContainerStarted","Data":"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"}
Oct 30 19:24:30.005016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.004593    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" podUID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" containerName="authorino" containerID="cri-o://1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5" gracePeriod=30
Oct 30 19:24:30.026665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.026622    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" podStartSLOduration=1.411606938 podStartE2EDuration="2.026609701s" totalImagesPullingTime="615.002763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:30.025185783 +0000 UTC m=+334.994826620" watchObservedRunningTime="2026-10-30 19:24:30.026609701 +0000 UTC m=+334.996250536"
Oct 30 19:24:30.257991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.257928    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:30.349387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.349357    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\") pod \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") "
Oct 30 19:24:30.349545 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.349466    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\" (UniqueName: \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\") pod \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\" (UID: \"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4\") "
Oct 30 19:24:30.351750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.351722    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7" pod "5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" (UID: "5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4"). InnerVolumeSpecName "kube-api-access-5q8l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:30.361984 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.361963    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert" pod "5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" (UID: "5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:24:30.450851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.450826    5033 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-tls-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.450851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:30.450849    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q8l7\" (UniqueName: \"kubernetes.io/projected/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4-kube-api-access-5q8l7\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:31.010346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.010312    5033 generic.go:356] "Generic (PLEG): container finished" podID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" containerID="1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5" exitCode=0
Oct 30 19:24:31.010829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.010360    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-bklgv"
Oct 30 19:24:31.010829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.010393    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" event={"ID":"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4","Type":"ContainerDied","Data":"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"}
Oct 30 19:24:31.010829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.010435    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-bklgv" event={"ID":"5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4","Type":"ContainerDied","Data":"2db4fa923bca1fd8160b5e8cc4aea1a80d949311c7fa5a5c50a5946e31610c8c"}
Oct 30 19:24:31.010829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.010455    5033 scope.go:122] "RemoveContainer" containerID="1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"
Oct 30 19:24:31.019484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.019469    5033 scope.go:122] "RemoveContainer" containerID="1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"
Oct 30 19:24:31.019772 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:24:31.019754    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5\": container with ID starting with 1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5 not found: ID does not exist" containerID="1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"
Oct 30 19:24:31.019835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.019781    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5"} err="failed to get container status \"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5\": rpc error: code = NotFound desc = could not find container \"1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5\": container with ID starting with 1f7230665ca02e5ea3f731fe65211cc9220aaeaa796a80ebd2dd0c002da789c5 not found: ID does not exist"
Oct 30 19:24:31.036670 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.036644    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:31.040895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.040869    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-bklgv"]
Oct 30 19:24:31.166963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:31.166944    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" path="/var/lib/kubelet/pods/5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4/volumes"
Oct 30 19:24:32.578968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.578935    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:24:32.579417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.579126    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" podUID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" containerName="authorino" containerID="cri-o://c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818" gracePeriod=30
Oct 30 19:24:32.839290 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.839238    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:24:32.871290 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.871268    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\" (UniqueName: \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\") pod \"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e\" (UID: \"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e\") "
Oct 30 19:24:32.873189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.873167    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc" pod "849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" (UID: "849b2d3e-c18f-49c5-b4f4-35c6f5c3454e"). InnerVolumeSpecName "kube-api-access-czhbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:32.973200 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:32.973165    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czhbc\" (UniqueName: \"kubernetes.io/projected/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e-kube-api-access-czhbc\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.022014 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.021976    5033 generic.go:356] "Generic (PLEG): container finished" podID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" containerID="c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818" exitCode=0
Oct 30 19:24:33.022210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.022033    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs"
Oct 30 19:24:33.022210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.022061    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" event={"ID":"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e","Type":"ContainerDied","Data":"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"}
Oct 30 19:24:33.022210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.022112    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-pksxs" event={"ID":"849b2d3e-c18f-49c5-b4f4-35c6f5c3454e","Type":"ContainerDied","Data":"fba4301c4bfd86c0947b60027eccd629192466655bf49ed6b4df7f8fe82cf1f8"}
Oct 30 19:24:33.022210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.022137    5033 scope.go:122] "RemoveContainer" containerID="c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"
Oct 30 19:24:33.031382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.031359    5033 scope.go:122] "RemoveContainer" containerID="c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"
Oct 30 19:24:33.031641 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:24:33.031619    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818\": container with ID starting with c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818 not found: ID does not exist" containerID="c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"
Oct 30 19:24:33.031711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.031651    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818"} err="failed to get container status \"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818\": rpc error: code = NotFound desc = could not find container \"c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818\": container with ID starting with c83aa2ffd48c0bfb17f6aaa0d0e0905596798333b82a8c5dee88b0c6180e7818 not found: ID does not exist"
Oct 30 19:24:33.045852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.045831    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:24:33.050906 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.050884    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-pksxs"]
Oct 30 19:24:33.167841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:33.167759    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" path="/var/lib/kubelet/pods/849b2d3e-c18f-49c5-b4f4-35c6f5c3454e/volumes"
Oct 30 19:24:34.985516 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:24:34.985485    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:24:53.729823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.729786    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"]
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730174    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" containerName="authorino"
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730185    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" containerName="authorino"
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730205    5033 cpu_manager.go:414] "containerMap: removing container" podUID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" containerName="authorino"
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730210    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" containerName="authorino"
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730273    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ac1ea27-5ca6-48fe-ae69-f51e26d49cc4" containerName="authorino"
Oct 30 19:24:53.730289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.730282    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="849b2d3e-c18f-49c5-b4f4-35c6f5c3454e" containerName="authorino"
Oct 30 19:24:53.733620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.733604    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.741338 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.741314    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"]
Oct 30 19:24:53.838107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.838079    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cvb8\" (UniqueName: \"kubernetes.io/projected/0e018cb3-2605-4a89-93d5-f27bc8522bf6-kube-api-access-2cvb8\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.838250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.838118    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e018cb3-2605-4a89-93d5-f27bc8522bf6-cert\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.939478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.939451    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cvb8\" (UniqueName: \"kubernetes.io/projected/0e018cb3-2605-4a89-93d5-f27bc8522bf6-kube-api-access-2cvb8\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.939612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.939496    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e018cb3-2605-4a89-93d5-f27bc8522bf6-cert\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.941827 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.941806    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e018cb3-2605-4a89-93d5-f27bc8522bf6-cert\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:53.949064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:53.949038    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cvb8\" (UniqueName: \"kubernetes.io/projected/0e018cb3-2605-4a89-93d5-f27bc8522bf6-kube-api-access-2cvb8\") pod \"llmisvc-controller-manager-6466c554d8-9mjnv\" (UID: \"0e018cb3-2605-4a89-93d5-f27bc8522bf6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:54.048460 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:54.048413    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:54.319114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:54.319091    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"]
Oct 30 19:24:54.320488 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:24:54.320454    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e018cb3_2605_4a89_93d5_f27bc8522bf6.slice/crio-221e5c6b16721505b1119e5f3912a27ce08c1aa70b1c132de86583acee4f4421 WatchSource:0}: Error finding container 221e5c6b16721505b1119e5f3912a27ce08c1aa70b1c132de86583acee4f4421: Status 404 returned error can't find the container with id 221e5c6b16721505b1119e5f3912a27ce08c1aa70b1c132de86583acee4f4421
Oct 30 19:24:55.135063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:55.134991    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv" event={"ID":"0e018cb3-2605-4a89-93d5-f27bc8522bf6","Type":"ContainerStarted","Data":"a9be18dc68c1cf206a17fdc1cc9d84ae49e683dc0cf9b9e02369a990bbc2d9ff"}
Oct 30 19:24:55.135063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:55.135051    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv" event={"ID":"0e018cb3-2605-4a89-93d5-f27bc8522bf6","Type":"ContainerStarted","Data":"221e5c6b16721505b1119e5f3912a27ce08c1aa70b1c132de86583acee4f4421"}
Oct 30 19:24:55.135447 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:55.135199    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:24:55.152901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:24:55.152854    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv" podStartSLOduration=1.720183727 podStartE2EDuration="2.152838865s" totalImagesPullingTime="432.655138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:55.15254584 +0000 UTC m=+360.122186675" watchObservedRunningTime="2026-10-30 19:24:55.152838865 +0000 UTC m=+360.122479701"
Oct 30 19:25:04.985018 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:25:04.984979    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:25:26.142114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.142079    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-9mjnv"
Oct 30 19:25:26.193030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.192992    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:25:26.193362 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.193320    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" podUID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" containerName="manager" containerID="cri-o://fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54" gracePeriod=30
Oct 30 19:25:26.437538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.437515    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:25:26.582422 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.582394    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\" (UniqueName: \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\") pod \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") "
Oct 30 19:25:26.582549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.582499    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\" (UniqueName: \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\") pod \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\" (UID: \"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8\") "
Oct 30 19:25:26.584335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.584310    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert" pod "901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" (UID: "901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:26.584554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.584533    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6" pod "901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" (UID: "901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8"). InnerVolumeSpecName "kube-api-access-q6fg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:26.683264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.683210    5033 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:26.683264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:26.683233    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6fg6\" (UniqueName: \"kubernetes.io/projected/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8-kube-api-access-q6fg6\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:27.289931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.289902    5033 generic.go:356] "Generic (PLEG): container finished" podID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" containerID="fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54" exitCode=0
Oct 30 19:25:27.290334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.289941    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"
Oct 30 19:25:27.290334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.289952    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" event={"ID":"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8","Type":"ContainerDied","Data":"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"}
Oct 30 19:25:27.290334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.290019    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2" event={"ID":"901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8","Type":"ContainerDied","Data":"79314041e3790f3a7546ad9c3601f6af9657a09fb2b31a132c9b19c2ab42a537"}
Oct 30 19:25:27.290334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.290053    5033 scope.go:122] "RemoveContainer" containerID="fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"
Oct 30 19:25:27.298457 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.298434    5033 scope.go:122] "RemoveContainer" containerID="fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"
Oct 30 19:25:27.298774 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:25:27.298752    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54\": container with ID starting with fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54 not found: ID does not exist" containerID="fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"
Oct 30 19:25:27.298839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.298781    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54"} err="failed to get container status \"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54\": rpc error: code = NotFound desc = could not find container \"fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54\": container with ID starting with fdf25670719747e9c3d577b3e18ae40b80d9dc27447cd75e7b88ab6e64412e54 not found: ID does not exist"
Oct 30 19:25:27.314024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.314005    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:25:27.320583 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:27.320565    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4xsh2"]
Oct 30 19:25:29.169548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:25:29.169516    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" path="/var/lib/kubelet/pods/901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8/volumes"
Oct 30 19:25:34.986009 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:25:34.985967    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:26:04.985553 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:26:04.985519    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:26:21.220294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.220260    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"]
Oct 30 19:26:21.220691 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.220662    5033 cpu_manager.go:414] "containerMap: removing container" podUID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" containerName="manager"
Oct 30 19:26:21.220691 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.220689    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" containerName="manager"
Oct 30 19:26:21.220784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.220774    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="901f3e83-8736-4eb5-91c8-c7e4f9ad3bd8" containerName="manager"
Oct 30 19:26:21.223913 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.223891    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.228239 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.228219    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tbg4v\""
Oct 30 19:26:21.237386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.237363    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"]
Oct 30 19:26:21.320594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320567    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320594    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320619    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320637    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320714    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320751    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f928a1-6d04-4493-82ff-d83fda077a36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320948 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320873    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320948 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320913    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.320948 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.320933    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcvj5\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-kube-api-access-vcvj5\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421807    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421849    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421869    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcvj5\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-kube-api-access-vcvj5\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421898    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421914    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.421940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421937    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421954    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421970    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.421988    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f928a1-6d04-4493-82ff-d83fda077a36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.422306    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.422383    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.422411    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.422602    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.422973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.422953    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f928a1-6d04-4493-82ff-d83fda077a36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.423908 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.423887    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.424375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.424359    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.432181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.432154    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcvj5\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-kube-api-access-vcvj5\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.432342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.432322    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f928a1-6d04-4493-82ff-d83fda077a36-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-sx9vv\" (UID: \"f0f928a1-6d04-4493-82ff-d83fda077a36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.536592 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.536541    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:21.815382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.815353    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"]
Oct 30 19:26:21.817423 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:26:21.817397    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f928a1_6d04_4493_82ff_d83fda077a36.slice/crio-d3700c298a8b2374c69e34944ca19cadb553d0631e6b2bffa7ddb70d3e1bbe63 WatchSource:0}: Error finding container d3700c298a8b2374c69e34944ca19cadb553d0631e6b2bffa7ddb70d3e1bbe63: Status 404 returned error can't find the container with id d3700c298a8b2374c69e34944ca19cadb553d0631e6b2bffa7ddb70d3e1bbe63
Oct 30 19:26:21.819498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.819468    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:21.819574 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.819528    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:21.819574 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:21.819558    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:22.579028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:22.578991    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" event={"ID":"f0f928a1-6d04-4493-82ff-d83fda077a36","Type":"ContainerStarted","Data":"2fbfe1d856b68126116713d6b13a5f17e5081ec1afa6d47b52949a5dbbad0d23"}
Oct 30 19:26:22.579453 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:22.579079    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" event={"ID":"f0f928a1-6d04-4493-82ff-d83fda077a36","Type":"ContainerStarted","Data":"d3700c298a8b2374c69e34944ca19cadb553d0631e6b2bffa7ddb70d3e1bbe63"}
Oct 30 19:26:22.603166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:22.603121    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podStartSLOduration=1.6031088690000002 podStartE2EDuration="1.603108869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:22.600959455 +0000 UTC m=+447.570600291" watchObservedRunningTime="2026-10-30 19:26:22.603108869 +0000 UTC m=+447.572749704"
Oct 30 19:26:23.537270 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:23.537241    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:23.538813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:23.538793    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:23.538868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:23.538847    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:24.537006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:24.536967    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:24.537475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:24.537043    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:25.537135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:25.537101    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:25.537511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:25.537161    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:26.537157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:26.537122    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:26.537562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:26.537183    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:27.537841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:27.537803    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:27.538262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:27.537856    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:28.537128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:28.537092    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:26:28.537293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:28.537158    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 19:26:29.963921 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:29.963891    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:29.964409 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:29.964279    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:29.965163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:29.965141    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv"
Oct 30 19:26:34.985837 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:26:34.985807    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:26:36.245880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.245836    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"]
Oct 30 19:26:36.249960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.249935    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.263842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.263814    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"]
Oct 30 19:26:36.284214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.284180    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"]
Oct 30 19:26:36.288127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.288107    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.300797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.300772    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"]
Oct 30 19:26:36.371710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371661    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.371710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371713    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5g2\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-kube-api-access-xf5g2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.371928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371782    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.371928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371810    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.371928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371847    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.372042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371962    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.372042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.371997    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.372112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.372045    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.372161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.372149    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.472520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472496    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.472648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472528    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.472648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472552    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.472648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472569    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.472788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472772    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.472834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472801    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5g2\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-kube-api-access-xf5g2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.472834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472825    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.472936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472850    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.472993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472927    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.472993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.472967    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473016    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.473097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473048    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473093    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmxc\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-kube-api-access-htmxc\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.473185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473132    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.473185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473165    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473222    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473231    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473293    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473320    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473352    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7dacba6-7983-4f68-8791-c46ed3bad49f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.473532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473416    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473515    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.473532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.473521    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.475316 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.475291    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.475663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.475646    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.481579 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.481557    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.481964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.481948    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5g2\" (UniqueName: \"kubernetes.io/projected/f93f9c61-295c-4fe6-a44d-6611ed4631a7-kube-api-access-xf5g2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-95d9v\" (UID: \"f93f9c61-295c-4fe6-a44d-6611ed4631a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.564178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.564130    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:36.574486 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574464    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7dacba6-7983-4f68-8791-c46ed3bad49f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574504    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574553    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574595    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574627    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574660    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574722    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574755    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmxc\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-kube-api-access-htmxc\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.574938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.574785    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.575404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.575126    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.575404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.575180    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.575404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.575215    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7dacba6-7983-4f68-8791-c46ed3bad49f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.575660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.575642    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.575740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.575701    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.576768 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.576744    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.577354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.577332    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.584221 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.584194    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.584447 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.584429    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmxc\" (UniqueName: \"kubernetes.io/projected/d7dacba6-7983-4f68-8791-c46ed3bad49f-kube-api-access-htmxc\") pod \"maas-default-gateway-openshift-default-75bf867b6c-8qdgp\" (UID: \"d7dacba6-7983-4f68-8791-c46ed3bad49f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.603366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.603344    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:36.842352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.842322    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"]
Oct 30 19:26:36.842531 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:26:36.842507    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93f9c61_295c_4fe6_a44d_6611ed4631a7.slice/crio-2a7d9dcd1ed62bdf497702658d9ef571b11cc1c917c4e031b9bd590bee208b98 WatchSource:0}: Error finding container 2a7d9dcd1ed62bdf497702658d9ef571b11cc1c917c4e031b9bd590bee208b98: Status 404 returned error can't find the container with id 2a7d9dcd1ed62bdf497702658d9ef571b11cc1c917c4e031b9bd590bee208b98
Oct 30 19:26:36.844958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.844923    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:36.845064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.844998    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:36.845064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.845044    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:36.888058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.888033    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"]
Oct 30 19:26:36.902669 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:26:36.902646    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7dacba6_7983_4f68_8791_c46ed3bad49f.slice/crio-cfb431a2ca546efdef79901f87bc76dacdc39b7275d789edf1904c1c9187d174 WatchSource:0}: Error finding container cfb431a2ca546efdef79901f87bc76dacdc39b7275d789edf1904c1c9187d174: Status 404 returned error can't find the container with id cfb431a2ca546efdef79901f87bc76dacdc39b7275d789edf1904c1c9187d174
Oct 30 19:26:36.904844 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.904814    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:36.904939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.904876    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:36.904939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:36.904903    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:26:37.691712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.690837    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" event={"ID":"d7dacba6-7983-4f68-8791-c46ed3bad49f","Type":"ContainerStarted","Data":"3b87c34c1681d5a7947547530ce1045bfd11b52f109a2a37701c210aa6426902"}
Oct 30 19:26:37.691712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.690918    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" event={"ID":"d7dacba6-7983-4f68-8791-c46ed3bad49f","Type":"ContainerStarted","Data":"cfb431a2ca546efdef79901f87bc76dacdc39b7275d789edf1904c1c9187d174"}
Oct 30 19:26:37.696477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.696447    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" event={"ID":"f93f9c61-295c-4fe6-a44d-6611ed4631a7","Type":"ContainerStarted","Data":"b8bee36dd9a95e67da985e455764287567cc48938af3919b03ea594398ca0246"}
Oct 30 19:26:37.696736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.696716    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" event={"ID":"f93f9c61-295c-4fe6-a44d-6611ed4631a7","Type":"ContainerStarted","Data":"2a7d9dcd1ed62bdf497702658d9ef571b11cc1c917c4e031b9bd590bee208b98"}
Oct 30 19:26:37.730615 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.730571    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podStartSLOduration=1.730558597 podStartE2EDuration="1.730558597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:37.728523679 +0000 UTC m=+462.698164515" watchObservedRunningTime="2026-10-30 19:26:37.730558597 +0000 UTC m=+462.700199431"
Oct 30 19:26:37.779700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:37.774035    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podStartSLOduration=1.774015369 podStartE2EDuration="1.774015369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:37.772158848 +0000 UTC m=+462.741799684" watchObservedRunningTime="2026-10-30 19:26:37.774015369 +0000 UTC m=+462.743656205"
Oct 30 19:26:38.570701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.567491    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:38.570701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.567646    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:38.570701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.567714    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:38.604104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.604079    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:38.605322 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.605293    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:38.605433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:38.605332    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:39.574075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:39.574036    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:39.574703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:39.574648    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:39.604275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:39.604240    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:39.604445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:39.604426    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:40.568186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:40.568152    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:40.568475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:40.568447    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:40.604295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:40.604260    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:40.604647 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:40.604301    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:41.580700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:41.575192    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:41.580700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:41.575255    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:41.603862 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:41.603832    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:41.604172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:41.604144    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:42.565592 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:42.565124    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:42.565592 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:42.565188    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:42.604142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:42.604108    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:42.604305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:42.604155    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:43.565403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:43.565363    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:26:43.565571 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:43.565419    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 19:26:43.603715 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:43.603660    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:26:43.604214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:43.603743    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 19:26:45.171040 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.171012    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:45.171582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.171398    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:45.172181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.172163    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v"
Oct 30 19:26:45.202275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.202254    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:45.202586 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.202551    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:26:45.203407 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:26:45.203390    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp"
Oct 30 19:27:04.985796 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:27:04.985761    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:27:27.989150 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:27.989119    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:27:27.993486 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:27.993465    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.005869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.005749    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-6cf459-openshift-default-dockercfg-ckp8r\""
Oct 30 19:27:28.009823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.009798    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:27:28.095588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.095556    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:27:28.100770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.100741    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.100896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.100843    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.100942 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.100892    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.100979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.100937    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.101012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.100975    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.101045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.101029    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfh9d\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.101078 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.101065    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.101111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.101078    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.101146 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.101125    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.101179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.101159    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.112637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.112614    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:27:28.144385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.141000    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-d55178-openshift-default-dockercfg-6lspn\""
Oct 30 19:27:28.202458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202430    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.202626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202471    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.202626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202517    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.202626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202549    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.202827 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202707    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.202827 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202782    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.202939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202845    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.202939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202880    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203040 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202977    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.203040 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.202913    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203046    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203331    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203615    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203758    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203805    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.203894 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203852    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.203894 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203880    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.204002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203925    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfh9d\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.204002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.203986    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.204104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.204028    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.204104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.204061    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfw5x\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.204104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.204095    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.204373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.204354    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.207225 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.207181    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.209140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.209115    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.218277 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.218252    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.225047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.225025    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfh9d\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.305590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305388    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.305590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305444    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfw5x\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.305590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305489    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.305590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305544    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.305590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305581    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305626    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305711    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305789    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305816    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.305868    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.306152    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.306490    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.306931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.306898    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.307086 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.307065    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.313287 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.309366    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.313287 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.311156    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:28.313879 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.313858    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.333695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.325742    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.343686 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.340173    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfw5x\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.422084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.421878    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:28.647875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.647832    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:27:28.650980 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:27:28.650951    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e029a6d_6838_4509_a872_fe1dbb27b219.slice/crio-cc2bf7ece57f9b13a5d479d2caf5df9d112c09aed2254110d2c9365884619ad9 WatchSource:0}: Error finding container cc2bf7ece57f9b13a5d479d2caf5df9d112c09aed2254110d2c9365884619ad9: Status 404 returned error can't find the container with id cc2bf7ece57f9b13a5d479d2caf5df9d112c09aed2254110d2c9365884619ad9
Oct 30 19:27:28.653795 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.653762    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.653950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.653840    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.653950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.653887    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.736438 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.736257    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:27:28.737324 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:27:28.737294    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0be903_95d1_475a_9c0c_d32b252c81a0.slice/crio-fed751f5a26f0a2ebfe4529ca8c7dafe67485174e1fbf0f2f8d24094a00034d4 WatchSource:0}: Error finding container fed751f5a26f0a2ebfe4529ca8c7dafe67485174e1fbf0f2f8d24094a00034d4: Status 404 returned error can't find the container with id fed751f5a26f0a2ebfe4529ca8c7dafe67485174e1fbf0f2f8d24094a00034d4
Oct 30 19:27:28.739902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.739854    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.740006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.739940    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.740006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.739985    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:28.961189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.961147    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerStarted","Data":"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"}
Oct 30 19:27:28.961378 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.961224    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerStarted","Data":"cc2bf7ece57f9b13a5d479d2caf5df9d112c09aed2254110d2c9365884619ad9"}
Oct 30 19:27:28.963552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.963515    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerStarted","Data":"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"}
Oct 30 19:27:28.963696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.963566    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerStarted","Data":"fed751f5a26f0a2ebfe4529ca8c7dafe67485174e1fbf0f2f8d24094a00034d4"}
Oct 30 19:27:28.985123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:28.985070    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podStartSLOduration=1.985051533 podStartE2EDuration="1.985051533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:28.98417061 +0000 UTC m=+513.953811446" watchObservedRunningTime="2026-10-30 19:27:28.985051533 +0000 UTC m=+513.954692370"
Oct 30 19:27:29.006609 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.006545    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podStartSLOduration=1.006529232 podStartE2EDuration="1.006529232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:29.005929192 +0000 UTC m=+513.975570029" watchObservedRunningTime="2026-10-30 19:27:29.006529232 +0000 UTC m=+513.976170068"
Oct 30 19:27:29.311992 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.311906    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:29.317757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.317734    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:29.424525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.424486    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:29.430360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.430331    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:29.969918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.969892    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:29.970181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.970144    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:29.970699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.970652    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:29.970922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:29.970904    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:34.985728 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:27:34.985686    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:27:35.024722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.024658    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:27:35.028663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.028637    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.039246 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.039203    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-171808\""
Oct 30 19:27:35.040565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.040541    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:27:35.060518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.060489    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:27:35.065004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.064980    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.066030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.066010    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-171808-dockercfg-t8rwz\""
Oct 30 19:27:35.086503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.086475    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:27:35.179361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.179326    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.179568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.179413    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxqlw\" (UniqueName: \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.179568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.179505    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.179728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.179572    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7m6h\" (UniqueName: \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.280651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.280549    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxqlw\" (UniqueName: \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.280651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.280603    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.280651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.280646    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7m6h\" (UniqueName: \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.280982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.280775    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.281765 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.281737    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.282000 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.281971    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.292550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.292526    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7m6h\" (UniqueName: \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\") pod \"payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.294353 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.294330    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxqlw\" (UniqueName: \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\") pod \"payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.347864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.347838    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:35.390714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.390451    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:35.676491 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.671805    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:27:35.725209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:35.725187    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:27:35.726854 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:27:35.726826    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cf5f80_c8f2_48ca_a4af_fe21cdb7dc34.slice/crio-344fc8cc5ee000d61e8b54f4705874b5b12d9abc4f4e95105f55866866f417a1 WatchSource:0}: Error finding container 344fc8cc5ee000d61e8b54f4705874b5b12d9abc4f4e95105f55866866f417a1: Status 404 returned error can't find the container with id 344fc8cc5ee000d61e8b54f4705874b5b12d9abc4f4e95105f55866866f417a1
Oct 30 19:27:36.007214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.007175    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" event={"ID":"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34","Type":"ContainerStarted","Data":"344fc8cc5ee000d61e8b54f4705874b5b12d9abc4f4e95105f55866866f417a1"}
Oct 30 19:27:36.008395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.008368    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" event={"ID":"904ebe95-7d4f-4d65-a224-98c169f90681","Type":"ContainerStarted","Data":"a4306138482d6d6502b031e15fdeab6f672b998303e15792c88d05ba07aa434c"}
Oct 30 19:27:36.112968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.112933    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:27:36.117902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.117880    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.139319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.139290    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:27:36.192637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.192602    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.192892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.192868    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm7cj\" (UniqueName: \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.220022 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.219998    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-639c15\""
Oct 30 19:27:36.222211 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.221409    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-639c15-dockercfg-2wzsf\""
Oct 30 19:27:36.297641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.295233    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm7cj\" (UniqueName: \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.297641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.295281    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.297641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.296573    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.308088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.308037    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm7cj\" (UniqueName: \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\") pod \"payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.446433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.446406    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:36.758659 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:36.758622    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:27:36.765758 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:27:36.765711    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4f240a_9663_46e3_8114_73c9b4ae0501.slice/crio-e73473e5e3090bbdc9a54dd8d60fc2f026b89a4c043683e22df33dc8d751755d WatchSource:0}: Error finding container e73473e5e3090bbdc9a54dd8d60fc2f026b89a4c043683e22df33dc8d751755d: Status 404 returned error can't find the container with id e73473e5e3090bbdc9a54dd8d60fc2f026b89a4c043683e22df33dc8d751755d
Oct 30 19:27:37.025567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:37.025407    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" event={"ID":"6c4f240a-9663-46e3-8114-73c9b4ae0501","Type":"ContainerStarted","Data":"e73473e5e3090bbdc9a54dd8d60fc2f026b89a4c043683e22df33dc8d751755d"}
Oct 30 19:27:37.921708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:37.920767    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:27:37.933335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:37.932368    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:37.996883 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:37.996833    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-9f2d57\""
Oct 30 19:27:37.997932 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:37.997907    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-9f2d57-dockercfg-b6jpw\""
Oct 30 19:27:38.039704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.038570    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.039704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.038619    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdrf7\" (UniqueName: \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.039704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.038885    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:27:38.139543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.139503    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.139742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.139548    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdrf7\" (UniqueName: \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.140342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.140312    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.188787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.185812    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdrf7\" (UniqueName: \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\") pod \"payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.258717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.255340    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:38.587352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:38.587320    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:27:39.061190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:39.061109    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" event={"ID":"ce29ab48-fb2c-47eb-b024-9e0744f9c869","Type":"ContainerStarted","Data":"0aa871957f1f3a933b78109345cddbb14f2cc56e8e43ff85aef6e07226d55006"}
Oct 30 19:27:40.072407 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.072358    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" event={"ID":"904ebe95-7d4f-4d65-a224-98c169f90681","Type":"ContainerStarted","Data":"47160548286b7a060863053e28d8386e9cd5cd228140c044c6aac3be6f00bf9f"}
Oct 30 19:27:40.073333 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.073308    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:40.078787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.077088    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" event={"ID":"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34","Type":"ContainerStarted","Data":"60b83c5026f0cc9bf0504a8fa2ca694e7cf3b48db88f3f7fc3e57eec19efec2c"}
Oct 30 19:27:40.078787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.077642    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:40.097028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.096965    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" podStartSLOduration=1.248888914 podStartE2EDuration="5.09694204s" totalImagesPullingTime="3.848053126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:40.096544027 +0000 UTC m=+525.066184864" watchObservedRunningTime="2026-10-30 19:27:40.09694204 +0000 UTC m=+525.066582875"
Oct 30 19:27:40.118714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:40.114943    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" podStartSLOduration=1.364983596 podStartE2EDuration="5.114927622s" totalImagesPullingTime="3.749944026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:40.114088366 +0000 UTC m=+525.083729203" watchObservedRunningTime="2026-10-30 19:27:40.114927622 +0000 UTC m=+525.084568459"
Oct 30 19:27:41.051507 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.051472    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:27:41.055375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.055351    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.070728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.067761    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:27:41.071061 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.071044    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:27:41.071618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.071231    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-898sh\""
Oct 30 19:27:41.071783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.071255    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-639c15-serving-cert\""
Oct 30 19:27:41.091317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.091287    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" event={"ID":"6c4f240a-9663-46e3-8114-73c9b4ae0501","Type":"ContainerStarted","Data":"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"}
Oct 30 19:27:41.091729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.091349    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:41.093081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.093047    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" event={"ID":"ce29ab48-fb2c-47eb-b024-9e0744f9c869","Type":"ContainerStarted","Data":"eeb5398f42908c8f2403175aa7f6ef1bc683aa3215ca294c0ba2853cd0a405d6"}
Oct 30 19:27:41.093176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.093104    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:41.121106 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.121068    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" podStartSLOduration=1.707383871 podStartE2EDuration="5.121050759s" totalImagesPullingTime="3.413666888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:41.118670501 +0000 UTC m=+526.088311336" watchObservedRunningTime="2026-10-30 19:27:41.121050759 +0000 UTC m=+526.090691585"
Oct 30 19:27:41.145829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.145789    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" podStartSLOduration=2.5254538909999997 podStartE2EDuration="4.145777648s" totalImagesPullingTime="1.620323757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:41.143384555 +0000 UTC m=+526.113025391" watchObservedRunningTime="2026-10-30 19:27:41.145777648 +0000 UTC m=+526.115418480"
Oct 30 19:27:41.185950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.185922    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.186072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.185964    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.186072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.185995    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twn62\" (UniqueName: \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.186214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.186188    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.287009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.286974    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.287581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.287278    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.287581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.287324    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.287581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.287361    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twn62\" (UniqueName: \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.288835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.288812    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.300772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.300744    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.309376 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.309288    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.310155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.310128    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twn62\" (UniqueName: \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\") pod \"maas-api-e2e-worker-w6-639c15-7688ff4654-prplw\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.378020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.377991    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:41.690154 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:41.690077    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:27:41.695507 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:27:41.695472    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84a3a0a_1edc_4e03_a4a5_1e7862b14bda.slice/crio-fa286e0ec55bb40bde92725e739b7708f841afded077e2dffbbd5a669dcc28f8 WatchSource:0}: Error finding container fa286e0ec55bb40bde92725e739b7708f841afded077e2dffbbd5a669dcc28f8: Status 404 returned error can't find the container with id fa286e0ec55bb40bde92725e739b7708f841afded077e2dffbbd5a669dcc28f8
Oct 30 19:27:42.100582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:42.100544    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" event={"ID":"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda","Type":"ContainerStarted","Data":"fa286e0ec55bb40bde92725e739b7708f841afded077e2dffbbd5a669dcc28f8"}
Oct 30 19:27:44.119963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:44.119927    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" event={"ID":"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda","Type":"ContainerStarted","Data":"5e7348cbd9d8604f5ffc67c595330a346a198d712b84e77b202aafa38792c933"}
Oct 30 19:27:44.120622 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:44.120605    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:44.148497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:44.148450    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" podStartSLOduration=1.554476421 podStartE2EDuration="3.148437955s" totalImagesPullingTime="1.593961534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:44.144282563 +0000 UTC m=+529.113923399" watchObservedRunningTime="2026-10-30 19:27:44.148437955 +0000 UTC m=+529.118078789"
Oct 30 19:27:51.138139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:51.138107    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:27:52.104502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:52.104470    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:27:52.104995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:52.104974    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:27:52.105171 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:52.105152    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:27:52.105276 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:52.105194    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:27:53.199145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.199112    5033 generic.go:356] "Generic (PLEG): container finished" podID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerID="cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06" exitCode=0
Oct 30 19:27:53.199645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.199191    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerDied","Data":"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"}
Oct 30 19:27:53.199645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.199620    5033 scope.go:122] "RemoveContainer" containerID="cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"
Oct 30 19:27:53.201156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201134    5033 generic.go:356] "Generic (PLEG): container finished" podID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerID="462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3" exitCode=0
Oct 30 19:27:53.201156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201140    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:53.201279 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201210    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:53.201279 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201256    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:53.201354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201211    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerDied","Data":"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"}
Oct 30 19:27:53.201643 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.201626    5033 scope.go:122] "RemoveContainer" containerID="462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"
Oct 30 19:27:53.202771 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.202744    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:53.202889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.202868    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:53.203005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:53.202905    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:54.132344 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.132311    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:27:54.139210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.139189    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.148468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.148439    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:27:54.148868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.148848    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-d55178\"/\"simulator-w2-d55178-kserve-self-signed-certs\""
Oct 30 19:27:54.149051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.149037    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-d55178\"/\"kube-root-ca.crt\""
Oct 30 19:27:54.149159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.149113    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-d55178\"/\"default-dockercfg-vtw2b\""
Oct 30 19:27:54.166729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.149226    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-d55178\"/\"openshift-service-ca.crt\""
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238208    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238298    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b4fg\" (UniqueName: \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238349    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238381    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238427    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.239215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.238480    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.333248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.333201    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerStarted","Data":"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"}
Oct 30 19:27:54.335775 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.335746    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerStarted","Data":"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"}
Oct 30 19:27:54.343337 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.343293    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.343982    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b4fg\" (UniqueName: \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.344037    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.344062    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.344107    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.344153    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.344601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.344577    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.345256 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.345227    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.345610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.345559    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.346369 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.346347    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.347639 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.347612    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.388743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.385272    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b4fg\" (UniqueName: \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\") pod \"simulator-w2-d55178-kserve-79ff794f47-gbkk4\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") " pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.437411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.437376    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:27:54.437783 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.437701    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:54.445742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.437935    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:54.644950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.644854    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:27:54.978279 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:54.978239    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:27:55.048391 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.048359    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:27:55.048762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.048740    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.068602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.066451    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"kube-root-ca.crt\""
Oct 30 19:27:55.068602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.067241    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"default-dockercfg-mtr5j\""
Oct 30 19:27:55.068602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.067331    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"openshift-service-ca.crt\""
Oct 30 19:27:55.068602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.067459    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"simulator-w0-171808-kserve-self-signed-certs\""
Oct 30 19:27:55.102696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.095493    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224242    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224340    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224452    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224497    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvwpk\" (UniqueName: \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224551    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.226026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.224583    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.321299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.321221    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:55.321921 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.321884    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:27:55.328807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.328763    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 19:27:55.344064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.330565    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvwpk\" (UniqueName: \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.344394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.344375    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.344544 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.344530    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.344796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.344783    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.344926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.344911    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.345241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.345212    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.349064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.348073    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.349064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.348344    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.349064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.348590    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.353962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.353326    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"simulator-w0-171808-kserve-self-signed-certs\""
Oct 30 19:27:55.353962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.353420    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"kube-root-ca.crt\""
Oct 30 19:27:55.371053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.370941    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"openshift-service-ca.crt\""
Oct 30 19:27:55.396729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.391928    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.403053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.402418    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" event={"ID":"9a667eb7-4853-46d8-bacb-e27de8464c1c","Type":"ContainerStarted","Data":"f4e2bd00494136c6e91569f99d57a97f2db0fca3b2f83e26105218e10bc15726"}
Oct 30 19:27:55.423539 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.423512    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.432656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.431193    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvwpk\" (UniqueName: \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\") pod \"simulator-w0-171808-kserve-669b5844f6-lrs4b\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") " pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:55.432656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.431762    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:55.432656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.431809    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:55.727749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.726783    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"default-dockercfg-mtr5j\""
Oct 30 19:27:55.734278 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:55.734041    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:27:56.208753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.204716    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:27:56.313503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.313059    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:27:56.313503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.313117    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 19:27:56.424531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.424101    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:56.424531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.424159    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:56.435543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:56.435503    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" event={"ID":"77d09205-17a9-4dfc-98d1-6440f6991154","Type":"ContainerStarted","Data":"79918fd22e3374a118478bedd7d3f32c4d6391f6c1bbd24e99918b58c9719b27"}
Oct 30 19:27:57.312598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:57.312555    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:27:57.312811 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:57.312624    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 19:27:57.424656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:57.424194    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:57.424656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:57.424263    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:58.329727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:58.315022    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:27:58.329727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:58.315083    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 19:27:58.433511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:58.433090    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:58.433511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:58.433156    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:59.315709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.312317    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:27:59.315709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.312375    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 19:27:59.424480 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.424438    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:27:59.424688 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.424569    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 19:27:59.971237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.970272    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:27:59.971237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:27:59.970351    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:28:00.324813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.324137    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:28:00.455249 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.455217    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:28:00.489748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.489716    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:28:00.490507 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.490486    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:28:00.694038 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.694008    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:28:00.720786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.720752    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.721707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.721656    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:28:00.730656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.730634    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"default-dockercfg-bhsf7\""
Oct 30 19:28:00.730790 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.730760    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"kube-root-ca.crt\""
Oct 30 19:28:00.730868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.730807    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"openshift-service-ca.crt\""
Oct 30 19:28:00.730987 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.730967    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3bc085\"/\"simulator-w4-3bc085-kserve-self-signed-certs\""
Oct 30 19:28:00.821720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821590    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.821720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821632    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9p4\" (UniqueName: \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.821720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821657    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.821903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821862    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.821939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821910    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.821978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.821955    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923206    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923261    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9p4\" (UniqueName: \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923299    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923390    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923428    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.923627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923467    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.924011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.923988    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.924905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.924242    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.924905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.924852    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.928468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.928445    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.928774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.928756    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:00.933810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:00.933787    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9p4\" (UniqueName: \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\") pod \"simulator-w4-3bc085-kserve-5c77d9798d-qq7nq\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") " pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:01.074152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.074110    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:01.361482 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.361455    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:28:01.362585 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:01.362549    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54432de9_313b_4824_b1f3_be4ca88764ef.slice/crio-47188d42b22192816aa1da91b5e27b7fc85bc567caeb79fda2e4c10e09d2eb3b WatchSource:0}: Error finding container 47188d42b22192816aa1da91b5e27b7fc85bc567caeb79fda2e4c10e09d2eb3b: Status 404 returned error can't find the container with id 47188d42b22192816aa1da91b5e27b7fc85bc567caeb79fda2e4c10e09d2eb3b
Oct 30 19:28:01.494381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.494341    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" event={"ID":"77d09205-17a9-4dfc-98d1-6440f6991154","Type":"ContainerStarted","Data":"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"}
Oct 30 19:28:01.496421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.496381    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" event={"ID":"54432de9-313b-4824-b1f3-be4ca88764ef","Type":"ContainerStarted","Data":"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"}
Oct 30 19:28:01.496670 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.496653    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" event={"ID":"54432de9-313b-4824-b1f3-be4ca88764ef","Type":"ContainerStarted","Data":"47188d42b22192816aa1da91b5e27b7fc85bc567caeb79fda2e4c10e09d2eb3b"}
Oct 30 19:28:01.498457 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.498416    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" event={"ID":"9a667eb7-4853-46d8-bacb-e27de8464c1c","Type":"ContainerStarted","Data":"c2ee3e12e051e383c1621549bd706a6750ae78c6a6d8ab6fec5aaf855af044b3"}
Oct 30 19:28:01.527181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.522612    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podStartSLOduration=3.07319242 podStartE2EDuration="7.52258963s" totalImagesPullingTime="4.44939721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:01.515121205 +0000 UTC m=+546.484762046" watchObservedRunningTime="2026-10-30 19:28:01.52258963 +0000 UTC m=+546.492230467"
Oct 30 19:28:01.539809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.539766    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podStartSLOduration=2.130766165 podStartE2EDuration="7.539754445s" totalImagesPullingTime="5.40898828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:01.537435743 +0000 UTC m=+546.507076575" watchObservedRunningTime="2026-10-30 19:28:01.539754445 +0000 UTC m=+546.509395280"
Oct 30 19:28:01.563136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.563097    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podStartSLOduration=1.5630832099999998 podStartE2EDuration="1.56308321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:01.562854667 +0000 UTC m=+546.532495502" watchObservedRunningTime="2026-10-30 19:28:01.56308321 +0000 UTC m=+546.532724106"
Oct 30 19:28:01.645002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.644947    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:28:01.735272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:01.735240    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:28:02.074964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:02.074934    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:04.645309 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:04.645276    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:28:04.660392 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:04.660367    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:28:04.991630 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:28:04.991593    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:28:05.535215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:05.535186    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:28:05.735743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:05.735710    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:28:05.748961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:05.748930    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:28:06.539027 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:06.538992    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:28:10.538058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.538023    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:28:10.542304 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.542279    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.553544 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.553515    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:28:10.554284 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.553951    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"simulator-w6-639c15-kserve-self-signed-certs\""
Oct 30 19:28:10.554506 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.554478    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"openshift-service-ca.crt\""
Oct 30 19:28:10.554651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.554509    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"default-dockercfg-jxglj\""
Oct 30 19:28:10.554771 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.554568    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"kube-root-ca.crt\""
Oct 30 19:28:10.629807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.629770    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.630002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.629815    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvkj\" (UniqueName: \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.630002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.629840    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.630120 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.629991    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.630120 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.630064    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.630228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.630131    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730668 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730632    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730738    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730791    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730831    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvkj\" (UniqueName: \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730864    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.730995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.730921    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.731122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.731099    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.731184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.731136    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.731276 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.731217    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.733181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.733151    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.733454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.733434    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.743331 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.743295    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvkj\" (UniqueName: \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\") pod \"simulator-w6-639c15-kserve-79489b87f8-6ph68\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") " pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:10.860737 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:10.860649    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:11.074432 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.074390    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:11.087621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.087600    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:11.147619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.147593    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:28:11.147907 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:11.147879    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode83d1981_e352_46a7_9cc3_a373943ab3d3.slice/crio-bb8b7cfcb787c5031840d37a39cd991daa8ab2bb204ca8f064c8c918fe2ed9f2 WatchSource:0}: Error finding container bb8b7cfcb787c5031840d37a39cd991daa8ab2bb204ca8f064c8c918fe2ed9f2: Status 404 returned error can't find the container with id bb8b7cfcb787c5031840d37a39cd991daa8ab2bb204ca8f064c8c918fe2ed9f2
Oct 30 19:28:11.565036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.565002    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" event={"ID":"e83d1981-e352-46a7-9cc3-a373943ab3d3","Type":"ContainerStarted","Data":"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"}
Oct 30 19:28:11.565036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.565048    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" event={"ID":"e83d1981-e352-46a7-9cc3-a373943ab3d3","Type":"ContainerStarted","Data":"bb8b7cfcb787c5031840d37a39cd991daa8ab2bb204ca8f064c8c918fe2ed9f2"}
Oct 30 19:28:11.578113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.578088    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:28:11.582752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.582383    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podStartSLOduration=1.582368585 podStartE2EDuration="1.582368585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:11.582225124 +0000 UTC m=+556.551865961" watchObservedRunningTime="2026-10-30 19:28:11.582368585 +0000 UTC m=+556.552009421"
Oct 30 19:28:11.870319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.870217    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:28:11.879916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.879889    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:28:11.879916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.879932    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:11.880144 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.880018    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.887279 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.887249    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"openshift-service-ca.crt\""
Oct 30 19:28:11.887917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.887884    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"default-dockercfg-jjz7n\""
Oct 30 19:28:11.888035 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.887921    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"kube-root-ca.crt\""
Oct 30 19:28:11.888104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.888054    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"simulator-w3-9f2d57-kserve-self-signed-certs\""
Oct 30 19:28:11.945283 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945254    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.945433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945297    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxkg\" (UniqueName: \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.945433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945332    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.945554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945455    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.945554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945527    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:11.945731 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:11.945707    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046509 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046479    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046519    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046543    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxkg\" (UniqueName: \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046567    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046599    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.046725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046661    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.047019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046929    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.047019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.046973    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.047113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.047023    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.048966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.048942    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.049143 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.049124    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.062696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.062539    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxkg\" (UniqueName: \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\") pod \"simulator-w3-9f2d57-kserve-78bb564b49-9plzr\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") " pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.200965 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.200941    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:12.484247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.484211    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:28:12.485604 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:12.485579    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e423276_0c16_418c_bbd3_5cc1a698cd98.slice/crio-ba878bc2fdb52ac36139c44d49d8ac24a58a9613a5c6966f1defd71a40a69c06 WatchSource:0}: Error finding container ba878bc2fdb52ac36139c44d49d8ac24a58a9613a5c6966f1defd71a40a69c06: Status 404 returned error can't find the container with id ba878bc2fdb52ac36139c44d49d8ac24a58a9613a5c6966f1defd71a40a69c06
Oct 30 19:28:12.574738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.574697    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" event={"ID":"4e423276-0c16-418c-bbd3-5cc1a698cd98","Type":"ContainerStarted","Data":"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"}
Oct 30 19:28:12.575141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.574773    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" event={"ID":"4e423276-0c16-418c-bbd3-5cc1a698cd98","Type":"ContainerStarted","Data":"ba878bc2fdb52ac36139c44d49d8ac24a58a9613a5c6966f1defd71a40a69c06"}
Oct 30 19:28:12.595693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:12.595058    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podStartSLOduration=1.595044606 podStartE2EDuration="1.595044606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:12.592456194 +0000 UTC m=+557.562097029" watchObservedRunningTime="2026-10-30 19:28:12.595044606 +0000 UTC m=+557.564685465"
Oct 30 19:28:13.201359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:13.201323    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:20.861024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:20.860984    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:20.880612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:20.880372    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:21.646989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:21.646959    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:28:22.201978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:22.201942    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:22.214406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:22.214384    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:22.648422 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:22.648340    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:28:25.554299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.554266    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:28:25.559175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.559149    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.567551 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.567527    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:28:25.567838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.567817    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"openshift-service-ca.crt\""
Oct 30 19:28:25.568593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.568572    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"kube-root-ca.crt\""
Oct 30 19:28:25.568741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.568726    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"default-dockercfg-jsz2z\""
Oct 30 19:28:25.568880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.568862    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"premium-w1-6cf459-kserve-self-signed-certs\""
Oct 30 19:28:25.705345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.705300    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qpph\" (UniqueName: \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.705702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.705661    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.705927 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.705910    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.706101 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.706083    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.706272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.706257    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.706408 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.706394    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808188    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808249    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808298    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808320    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808502    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qpph\" (UniqueName: \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808533    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808653    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.808966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808924    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.809109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.808983    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.810800 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.810776    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.811186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.811165    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.824064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.824041    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qpph\" (UniqueName: \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\") pod \"premium-w1-6cf459-kserve-6f6bf66644-8cr4j\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") " pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:25.881989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:25.881966    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:26.192753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:26.192709    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:28:26.193672 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:26.193647    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae35af12_8cd7_47bb_8031_0bfd1412b590.slice/crio-5a44cded5415d2b45b4c33c1903f7673f70fe49e0b6a981c4482aca820f77dd3 WatchSource:0}: Error finding container 5a44cded5415d2b45b4c33c1903f7673f70fe49e0b6a981c4482aca820f77dd3: Status 404 returned error can't find the container with id 5a44cded5415d2b45b4c33c1903f7673f70fe49e0b6a981c4482aca820f77dd3
Oct 30 19:28:26.668308 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:26.668270    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" event={"ID":"ae35af12-8cd7-47bb-8031-0bfd1412b590","Type":"ContainerStarted","Data":"e45a7e9235e9b321a036b92047806c06b74a059fd5423fbcccc39d7c675fd8d6"}
Oct 30 19:28:26.668733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:26.668329    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" event={"ID":"ae35af12-8cd7-47bb-8031-0bfd1412b590","Type":"ContainerStarted","Data":"5a44cded5415d2b45b4c33c1903f7673f70fe49e0b6a981c4482aca820f77dd3"}
Oct 30 19:28:26.692896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:26.692852    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podStartSLOduration=1.692840962 podStartE2EDuration="1.692840962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:26.691768815 +0000 UTC m=+571.661409651" watchObservedRunningTime="2026-10-30 19:28:26.692840962 +0000 UTC m=+571.662481796"
Oct 30 19:28:26.882784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:26.882751    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:34.990781 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:28:34.990737    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:28:35.882205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:35.882116    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:35.895892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:35.895868    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:36.751649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:36.751620    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:28:43.053426 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.053387    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:28:43.056763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.056740    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.079801 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.079774    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-171808-openshift-default-dockercfg-mzssz\""
Oct 30 19:28:43.089613 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.089588    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:28:43.091173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091142    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091280 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091208    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091280 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091267    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091309    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091327    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091360    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq4xc\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091388    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091426    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.091606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.091453    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192380    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192443    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192537    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192581    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192605    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192630    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192646    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.192716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192693    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4xc\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.193264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.192723    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.193264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.193165    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.193264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.193213    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.194343 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.194300    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.194520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.194498    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.194791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.194767    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.195750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.195725    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.196621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.196356    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.210555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.210528    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.210630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.210531    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq4xc\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.371611 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.371536    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:43.668688 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.668648    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:28:43.671304 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:43.671271    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd61b096_84b3_4bb1_a3f1_dd1c36cf5ef9.slice/crio-ecc83d3ad911f808242e40feacd1a145f5735cd3f53e81048db617798e910029 WatchSource:0}: Error finding container ecc83d3ad911f808242e40feacd1a145f5735cd3f53e81048db617798e910029: Status 404 returned error can't find the container with id ecc83d3ad911f808242e40feacd1a145f5735cd3f53e81048db617798e910029
Oct 30 19:28:43.673938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.673901    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.674068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.673983    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.674068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.674026    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.786484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.786451    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" event={"ID":"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9","Type":"ContainerStarted","Data":"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"}
Oct 30 19:28:43.786753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.786730    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" event={"ID":"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9","Type":"ContainerStarted","Data":"ecc83d3ad911f808242e40feacd1a145f5735cd3f53e81048db617798e910029"}
Oct 30 19:28:43.815969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:43.815917    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podStartSLOduration=0.815894164 podStartE2EDuration="815.894164ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:43.81506652 +0000 UTC m=+588.784707357" watchObservedRunningTime="2026-10-30 19:28:43.815894164 +0000 UTC m=+588.785535002"
Oct 30 19:28:44.373700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:44.372877    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:44.382697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:44.378830    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:44.382697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:44.378919    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:45.376702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:45.371998    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:45.376702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:45.372075    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:46.381763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:46.373008    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:46.381763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:46.373085    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:47.372097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:47.372055    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:47.372393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:47.372362    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:48.012380 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.012345    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:28:48.017792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.017765    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.024159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.023930    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:28:48.029901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.029880    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"premium-w3-9f2d57-kserve-self-signed-certs\""
Oct 30 19:28:48.067396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.067360    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.067652 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.067633    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.067829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.067812    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.067942 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.067927    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvqj\" (UniqueName: \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.068147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.068131    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.068330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.068314    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.169360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.169313    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.170148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.170124    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.170345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.170329    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.170443 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.170431    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvqj\" (UniqueName: \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.170640 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.170625    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.170833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.170812    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.180693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.171320    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.180865 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.175264    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.180969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.175579    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.197363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.197331    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.204605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.201207    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.205417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.205393    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvqj\" (UniqueName: \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\") pod \"premium-w3-9f2d57-kserve-7c747f6f96-fpbtd\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") " pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.334777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.334672    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:48.374075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.374044    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:48.374313 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.374283    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:48.646919 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.646883    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:28:48.647460 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:48.647431    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3feecb03_3ef3_4546_876d_7b6d079a202c.slice/crio-2668583053b3543e03729f319edf03073b331fdcf6a219f1faab7d85c6aa9eff WatchSource:0}: Error finding container 2668583053b3543e03729f319edf03073b331fdcf6a219f1faab7d85c6aa9eff: Status 404 returned error can't find the container with id 2668583053b3543e03729f319edf03073b331fdcf6a219f1faab7d85c6aa9eff
Oct 30 19:28:48.829893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.829865    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" event={"ID":"3feecb03-3ef3-4546-876d-7b6d079a202c","Type":"ContainerStarted","Data":"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"}
Oct 30 19:28:48.829986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.829906    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" event={"ID":"3feecb03-3ef3-4546-876d-7b6d079a202c","Type":"ContainerStarted","Data":"2668583053b3543e03729f319edf03073b331fdcf6a219f1faab7d85c6aa9eff"}
Oct 30 19:28:48.886701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:48.886455    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podStartSLOduration=1.8864324670000001 podStartE2EDuration="1.886432467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:48.864042833 +0000 UTC m=+593.833683668" watchObservedRunningTime="2026-10-30 19:28:48.886432467 +0000 UTC m=+593.856073303"
Oct 30 19:28:49.335242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:49.335201    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:49.375079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:49.375046    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:28:49.375340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:49.375313    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 19:28:50.426824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:50.426789    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:50.427704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:50.427640    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:50.428641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:50.428619    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:28:55.236968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:55.236938    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:28:55.238517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:55.238494    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:28:57.755005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.754958    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:28:57.759989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.759969    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.774546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.774524    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3bc085-openshift-default-dockercfg-6wbgq\""
Oct 30 19:28:57.776107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.776084    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:28:57.819178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819148    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819199    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819329    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819418    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819472    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819509    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxfwj\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819577    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819604    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.819642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.819638    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.920797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.920767    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.920977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.920824    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.920977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.920865    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.920977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.920900    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921046    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921090    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxfwj\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921142    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921180    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921242    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921418 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921390    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921478    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921576    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921761 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921739    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.921961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.921942    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.923435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.923406    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.923525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.923460    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.938484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.938453    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:57.938592 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:57.938517    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxfwj\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:58.073669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.073589    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:58.335504 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.335356    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:58.352815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.352789    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:58.372857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.372755    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:28:58.375804 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:28:58.375781    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79091d89_ccf8_489e_beaf_fcf36c91f684.slice/crio-7a274d24582cce92a1f5eaff830f723e46b8177370aa9a839029aab5c76dd68d WatchSource:0}: Error finding container 7a274d24582cce92a1f5eaff830f723e46b8177370aa9a839029aab5c76dd68d: Status 404 returned error can't find the container with id 7a274d24582cce92a1f5eaff830f723e46b8177370aa9a839029aab5c76dd68d
Oct 30 19:28:58.378294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.378260    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.378406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.378342    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.378406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.378385    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.901480 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.901444    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" event={"ID":"79091d89-ccf8-489e-beaf-fcf36c91f684","Type":"ContainerStarted","Data":"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"}
Oct 30 19:28:58.901990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.901514    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" event={"ID":"79091d89-ccf8-489e-beaf-fcf36c91f684","Type":"ContainerStarted","Data":"7a274d24582cce92a1f5eaff830f723e46b8177370aa9a839029aab5c76dd68d"}
Oct 30 19:28:58.915534 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.915504    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:28:58.930751 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:58.930710    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podStartSLOduration=1.930696626 podStartE2EDuration="1.930696626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:58.929379575 +0000 UTC m=+603.899020409" watchObservedRunningTime="2026-10-30 19:28:58.930696626 +0000 UTC m=+603.900337458"
Oct 30 19:28:59.081145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:59.075000    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:28:59.081791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:59.081445    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:28:59.081791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:28:59.081505    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:00.075357 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:00.075256    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:29:00.075357 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:00.075325    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:01.078896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:01.078854    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:29:01.079501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:01.079472    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:02.075515 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:02.075037    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:29:02.075515 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:02.075112    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:03.074705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:03.074140    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:29:03.074705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:03.074237    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:04.080173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:04.080139    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:29:04.080808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:04.080774    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 19:29:04.987820 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:29:04.987779    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:29:05.078985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:05.078954    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:29:05.079347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:05.079324    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:29:05.080056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:05.080034    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:29:24.141806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.141770    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"]
Oct 30 19:29:24.147740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.147717    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.152265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.152224    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:29:24.154714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.153066    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"]
Oct 30 19:29:24.154714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.153395    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:29:24.239406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.239373    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.239537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.239434    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.239585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.239550    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7jd\" (UniqueName: \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.341156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.341125    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.341339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.341174    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.341339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.341195    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7jd\" (UniqueName: \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.341917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.341896    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.343629 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.343604    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.353931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.353897    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7jd\" (UniqueName: \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\") pod \"maas-api-revoke-sub-4a7202ab-l99pw\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.471372 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.471340    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:24.770098 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:24.770038    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f6a870_cce1_4ad3_9212_ae1cbf722794.slice/crio-324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5 WatchSource:0}: Error finding container 324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5: Status 404 returned error can't find the container with id 324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5
Oct 30 19:29:24.770489 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:24.770463    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"]
Oct 30 19:29:25.067264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:25.067183    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw" event={"ID":"d3f6a870-cce1-4ad3-9212-ae1cbf722794","Type":"ContainerStarted","Data":"324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5"}
Oct 30 19:29:28.091682 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.091652    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw" event={"ID":"d3f6a870-cce1-4ad3-9212-ae1cbf722794","Type":"ContainerStarted","Data":"230ba7f0cab01b2d129c90134ce03f8b8edc5dd4ec0c28c57aeba18495a10e18"}
Oct 30 19:29:28.128628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.128584    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw" podStartSLOduration=1.124569621 podStartE2EDuration="4.12856963s" totalImagesPullingTime="3.004000009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:28.126216718 +0000 UTC m=+633.095857554" watchObservedRunningTime="2026-10-30 19:29:28.12856963 +0000 UTC m=+633.098210464"
Oct 30 19:29:28.293016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.292986    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"]
Oct 30 19:29:28.297101 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.297082    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.310913 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.310860    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9xqh\" (UniqueName: \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.311002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.310914    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.311055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.311028    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.318188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.317850    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"]
Oct 30 19:29:28.412172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.412137    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.412356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.412194    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.412356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.412334    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9xqh\" (UniqueName: \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.413066 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.413037    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.414959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.414917    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.425046 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.425022    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9xqh\" (UniqueName: \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\") pod \"maas-api-revoke-sub-e313cb88-dnmf7\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.611267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.611183    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:28.908825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:28.908482    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"]
Oct 30 19:29:28.911875 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:28.911845    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8edcdf_ddb6_4cb9_81e3_2719d0dd765d.slice/crio-f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6 WatchSource:0}: Error finding container f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6: Status 404 returned error can't find the container with id f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6
Oct 30 19:29:29.100506 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:29.100470    5033 generic.go:356] "Generic (PLEG): container finished" podID="d3f6a870-cce1-4ad3-9212-ae1cbf722794" containerID="230ba7f0cab01b2d129c90134ce03f8b8edc5dd4ec0c28c57aeba18495a10e18" exitCode=0
Oct 30 19:29:29.100953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:29.100557    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw" event={"ID":"d3f6a870-cce1-4ad3-9212-ae1cbf722794","Type":"ContainerDied","Data":"230ba7f0cab01b2d129c90134ce03f8b8edc5dd4ec0c28c57aeba18495a10e18"}
Oct 30 19:29:29.101880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:29.101854    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7" event={"ID":"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d","Type":"ContainerStarted","Data":"f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6"}
Oct 30 19:29:30.111896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.111864    5033 generic.go:356] "Generic (PLEG): container finished" podID="bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" containerID="5ceaea127ab017d5d67b3157417045a5c0f921d64d301df746f820f22a9ff5ec" exitCode=0
Oct 30 19:29:30.112391 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.112272    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7" event={"ID":"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d","Type":"ContainerDied","Data":"5ceaea127ab017d5d67b3157417045a5c0f921d64d301df746f820f22a9ff5ec"}
Oct 30 19:29:30.269610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.269578    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:30.336232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.336202    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\" (UniqueName: \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\") pod \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") "
Oct 30 19:29:30.336400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.336309    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\") pod \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") "
Oct 30 19:29:30.336400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.336363    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\") pod \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\" (UID: \"d3f6a870-cce1-4ad3-9212-ae1cbf722794\") "
Oct 30 19:29:30.336734 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.336710    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca" pod "d3f6a870-cce1-4ad3-9212-ae1cbf722794" (UID: "d3f6a870-cce1-4ad3-9212-ae1cbf722794"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:30.338837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.338810    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca" pod "d3f6a870-cce1-4ad3-9212-ae1cbf722794" (UID: "d3f6a870-cce1-4ad3-9212-ae1cbf722794"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:30.338953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.338936    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd" pod "d3f6a870-cce1-4ad3-9212-ae1cbf722794" (UID: "d3f6a870-cce1-4ad3-9212-ae1cbf722794"). InnerVolumeSpecName "kube-api-access-9j7jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:30.437813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.437746    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j7jd\" (UniqueName: \"kubernetes.io/projected/d3f6a870-cce1-4ad3-9212-ae1cbf722794-kube-api-access-9j7jd\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:30.437813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.437771    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:30.437813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:30.437782    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3f6a870-cce1-4ad3-9212-ae1cbf722794-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:31.120310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.120279    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"
Oct 30 19:29:31.120310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.120299    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw" event={"ID":"d3f6a870-cce1-4ad3-9212-ae1cbf722794","Type":"ContainerDied","Data":"324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5"}
Oct 30 19:29:31.120798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.120356    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="324b17029d050920aa0185f46ca98379eeeca3eca840bc4001a6ce8f64f742d5"
Oct 30 19:29:31.310191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.310165    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:31.348823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.348718    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\") pod \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") "
Oct 30 19:29:31.348963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.348858    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\" (UniqueName: \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\") pod \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") "
Oct 30 19:29:31.349043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.349006    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\") pod \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\" (UID: \"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d\") "
Oct 30 19:29:31.353713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.349289    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca" pod "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" (UID: "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:31.353713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.349580    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:31.353713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.351238    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca" pod "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" (UID: "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:31.353713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.352799    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh" pod "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" (UID: "bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d"). InnerVolumeSpecName "kube-api-access-s9xqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:31.450990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.450967    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9xqh\" (UniqueName: \"kubernetes.io/projected/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-kube-api-access-s9xqh\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:31.451111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:31.450992    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:32.127789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:32.127762    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"
Oct 30 19:29:32.128191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:32.127787    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7" event={"ID":"bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d","Type":"ContainerDied","Data":"f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6"}
Oct 30 19:29:32.128191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:32.127836    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3e5949e19cfb5bae9b26a373c92c3a1f11aa6febee61f17a49d3207cffc36c6"
Oct 30 19:29:34.988565 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:29:34.988529    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:29:36.589225 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589193    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"]
Oct 30 19:29:36.589764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589755    5033 cpu_manager.go:414] "containerMap: removing container" podUID="d3f6a870-cce1-4ad3-9212-ae1cbf722794" containerName="revoke-keys"
Oct 30 19:29:36.589842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589769    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3f6a870-cce1-4ad3-9212-ae1cbf722794" containerName="revoke-keys"
Oct 30 19:29:36.589842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589795    5033 cpu_manager.go:414] "containerMap: removing container" podUID="bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" containerName="revoke-keys"
Oct 30 19:29:36.589842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589799    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" containerName="revoke-keys"
Oct 30 19:29:36.589986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589883    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" containerName="revoke-keys"
Oct 30 19:29:36.589986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.589895    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3f6a870-cce1-4ad3-9212-ae1cbf722794" containerName="revoke-keys"
Oct 30 19:29:36.593445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.593423    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.599031 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.599007    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:29:36.599187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.599115    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:29:36.602001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.601546    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"]
Oct 30 19:29:36.710853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.710824    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.711016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.710953    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqsgq\" (UniqueName: \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.711075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.711032    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.812019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.811989    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.812184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.812039    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.812184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.812115    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqsgq\" (UniqueName: \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.812824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.812797    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.814383 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.814359    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.821041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.821015    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqsgq\" (UniqueName: \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\") pod \"maas-api-revoke-sub-f57864d9-d8rql\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:36.909790 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:36.909719    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:37.188567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:37.188540    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"]
Oct 30 19:29:37.191622 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:37.191595    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc201a1_c12d_4c5a_a738_a86972bb252b.slice/crio-1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb WatchSource:0}: Error finding container 1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb: Status 404 returned error can't find the container with id 1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb
Oct 30 19:29:38.168361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:38.168327    5033 generic.go:356] "Generic (PLEG): container finished" podID="1fc201a1-c12d-4c5a-a738-a86972bb252b" containerID="a00172ffb2b0f6fe89940c6e242d128d7f6f3ed60dc7e5e4c7d331ae14b3b44a" exitCode=0
Oct 30 19:29:38.168780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:38.168408    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql" event={"ID":"1fc201a1-c12d-4c5a-a738-a86972bb252b","Type":"ContainerDied","Data":"a00172ffb2b0f6fe89940c6e242d128d7f6f3ed60dc7e5e4c7d331ae14b3b44a"}
Oct 30 19:29:38.168780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:38.168493    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql" event={"ID":"1fc201a1-c12d-4c5a-a738-a86972bb252b","Type":"ContainerStarted","Data":"1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb"}
Oct 30 19:29:39.334366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.334336    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:39.444205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.444132    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\") pod \"1fc201a1-c12d-4c5a-a738-a86972bb252b\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") "
Oct 30 19:29:39.444205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.444199    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\") pod \"1fc201a1-c12d-4c5a-a738-a86972bb252b\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") "
Oct 30 19:29:39.444386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.444283    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\" (UniqueName: \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\") pod \"1fc201a1-c12d-4c5a-a738-a86972bb252b\" (UID: \"1fc201a1-c12d-4c5a-a738-a86972bb252b\") "
Oct 30 19:29:39.444587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.444551    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca" pod "1fc201a1-c12d-4c5a-a738-a86972bb252b" (UID: "1fc201a1-c12d-4c5a-a738-a86972bb252b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:39.444866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.444842    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:39.446403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.446374    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca" pod "1fc201a1-c12d-4c5a-a738-a86972bb252b" (UID: "1fc201a1-c12d-4c5a-a738-a86972bb252b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:39.446506 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.446457    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq" pod "1fc201a1-c12d-4c5a-a738-a86972bb252b" (UID: "1fc201a1-c12d-4c5a-a738-a86972bb252b"). InnerVolumeSpecName "kube-api-access-fqsgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:39.546237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.546208    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fc201a1-c12d-4c5a-a738-a86972bb252b-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:39.546237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.546234    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqsgq\" (UniqueName: \"kubernetes.io/projected/1fc201a1-c12d-4c5a-a738-a86972bb252b-kube-api-access-fqsgq\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:39.906951 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.906916    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"]
Oct 30 19:29:39.907445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.907431    5033 cpu_manager.go:414] "containerMap: removing container" podUID="1fc201a1-c12d-4c5a-a738-a86972bb252b" containerName="revoke-keys"
Oct 30 19:29:39.907510 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.907447    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fc201a1-c12d-4c5a-a738-a86972bb252b" containerName="revoke-keys"
Oct 30 19:29:39.907559 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.907552    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fc201a1-c12d-4c5a-a738-a86972bb252b" containerName="revoke-keys"
Oct 30 19:29:39.911123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.911104    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:39.923016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:39.922990    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"]
Oct 30 19:29:40.052043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.052014    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.052043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.052053    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vvr2\" (UniqueName: \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.052277 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.052173    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.094573 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.094542    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"]
Oct 30 19:29:40.099352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.099331    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.118747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.118722    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"]
Oct 30 19:29:40.153617 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.153574    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.153784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.153737    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vvr2\" (UniqueName: \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.153858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.153837    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.154304 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.154284    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.156338 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.156311    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.164153 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.164105    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vvr2\" (UniqueName: \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\") pod \"maas-api-revoke-sub-cf7b219e-lqrmn\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.181645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.181624    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"
Oct 30 19:29:40.181645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.181627    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql" event={"ID":"1fc201a1-c12d-4c5a-a738-a86972bb252b","Type":"ContainerDied","Data":"1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb"}
Oct 30 19:29:40.181784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.181666    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fa2c46ab1eca343eeeaadf6dc5d9fbf402de28c85d74a75a81a86d5128992bb"
Oct 30 19:29:40.227732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.227710    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:40.266950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.262476    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.266950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.262596    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.266950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.262644    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.366905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.365361    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.366905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.365429    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.366905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.365595    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.366905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.366460    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.382078 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.382027    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.382804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.382781    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a085c67d-h2lcb\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.415648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.415408    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:40.584205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.584174    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"]
Oct 30 19:29:40.718040 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:40.717798    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d93145_b8ef_4765_9bbf_887180ac277b.slice/crio-1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004 WatchSource:0}: Error finding container 1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004: Status 404 returned error can't find the container with id 1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004
Oct 30 19:29:40.724580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:40.724551    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"]
Oct 30 19:29:41.190949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.190916    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn" event={"ID":"77d7f8dc-2240-416a-ad87-59536dd9bf70","Type":"ContainerStarted","Data":"3a6d5bf33e0c9ab4e16dedfd2738348684476a92847ba6de0f4de1d2605a29da"}
Oct 30 19:29:41.191109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.190974    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn" event={"ID":"77d7f8dc-2240-416a-ad87-59536dd9bf70","Type":"ContainerStarted","Data":"146078e177b1601ad067e6870988dd31aa319e7321ab1f14a62f75ab5a800fb2"}
Oct 30 19:29:41.192738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.192715    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb" event={"ID":"e5d93145-b8ef-4765-9bbf-887180ac277b","Type":"ContainerStarted","Data":"83420c8a9c451db7e94dbce695399e9d5f64b28cededc0abfd099eaea23d1af8"}
Oct 30 19:29:41.192856 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.192751    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb" event={"ID":"e5d93145-b8ef-4765-9bbf-887180ac277b","Type":"ContainerStarted","Data":"1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004"}
Oct 30 19:29:41.211390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.211344    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn" podStartSLOduration=2.211330914 podStartE2EDuration="2.211330914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:41.209620069 +0000 UTC m=+646.179260904" watchObservedRunningTime="2026-10-30 19:29:41.211330914 +0000 UTC m=+646.180971749"
Oct 30 19:29:41.233929 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:41.233842    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb" podStartSLOduration=1.233830303 podStartE2EDuration="1.233830303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:41.231341947 +0000 UTC m=+646.200982781" watchObservedRunningTime="2026-10-30 19:29:41.233830303 +0000 UTC m=+646.203471138"
Oct 30 19:29:42.200859 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.200823    5033 generic.go:356] "Generic (PLEG): container finished" podID="77d7f8dc-2240-416a-ad87-59536dd9bf70" containerID="3a6d5bf33e0c9ab4e16dedfd2738348684476a92847ba6de0f4de1d2605a29da" exitCode=0
Oct 30 19:29:42.201288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.200904    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn" event={"ID":"77d7f8dc-2240-416a-ad87-59536dd9bf70","Type":"ContainerDied","Data":"3a6d5bf33e0c9ab4e16dedfd2738348684476a92847ba6de0f4de1d2605a29da"}
Oct 30 19:29:42.202908 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.202879    5033 generic.go:356] "Generic (PLEG): container finished" podID="e5d93145-b8ef-4765-9bbf-887180ac277b" containerID="83420c8a9c451db7e94dbce695399e9d5f64b28cededc0abfd099eaea23d1af8" exitCode=0
Oct 30 19:29:42.203032 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.202949    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb" event={"ID":"e5d93145-b8ef-4765-9bbf-887180ac277b","Type":"ContainerDied","Data":"83420c8a9c451db7e94dbce695399e9d5f64b28cededc0abfd099eaea23d1af8"}
Oct 30 19:29:42.892965 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.892912    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:29:42.897955 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.897929    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.912711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.912616    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:29:42.941636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.941612    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.941789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.941661    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.941789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.941766    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.941907 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.941881    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.941960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.941910    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.942017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.942004    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.942098 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.942080    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.942160 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.942130    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.942241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.942216    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfs97\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:42.954425 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:42.954398    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-f7f075-openshift-default-dockercfg-p74l2\""
Oct 30 19:29:43.043547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043514    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043552    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043585    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043599    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043634    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043709    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043758    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043797    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfs97\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.043889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043878    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.044121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.043957    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.044180 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.044112    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.044235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.044209    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.044377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.044353    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.044557 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.044539    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.046019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.045994    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.046561 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.046536    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.054371 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.054345    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.054496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.054479    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfs97\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.220071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.220042    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:43.396532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.396499    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:43.426917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.426890    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:43.448524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448332    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\") pod \"77d7f8dc-2240-416a-ad87-59536dd9bf70\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") "
Oct 30 19:29:43.448524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448375    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\") pod \"e5d93145-b8ef-4765-9bbf-887180ac277b\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") "
Oct 30 19:29:43.448524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448430    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\") pod \"e5d93145-b8ef-4765-9bbf-887180ac277b\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") "
Oct 30 19:29:43.448524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448465    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\" (UniqueName: \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\") pod \"77d7f8dc-2240-416a-ad87-59536dd9bf70\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") "
Oct 30 19:29:43.448849 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448658    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\") pod \"77d7f8dc-2240-416a-ad87-59536dd9bf70\" (UID: \"77d7f8dc-2240-416a-ad87-59536dd9bf70\") "
Oct 30 19:29:43.448849 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448828    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca" pod "e5d93145-b8ef-4765-9bbf-887180ac277b" (UID: "e5d93145-b8ef-4765-9bbf-887180ac277b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:43.448963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.448835    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\") pod \"e5d93145-b8ef-4765-9bbf-887180ac277b\" (UID: \"e5d93145-b8ef-4765-9bbf-887180ac277b\") "
Oct 30 19:29:43.449458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.449386    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:43.449875 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.449849    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca" pod "77d7f8dc-2240-416a-ad87-59536dd9bf70" (UID: "77d7f8dc-2240-416a-ad87-59536dd9bf70"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:43.451547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.451519    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q" pod "e5d93145-b8ef-4765-9bbf-887180ac277b" (UID: "e5d93145-b8ef-4765-9bbf-887180ac277b"). InnerVolumeSpecName "kube-api-access-72j8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:43.452423 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.452390    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca" pod "e5d93145-b8ef-4765-9bbf-887180ac277b" (UID: "e5d93145-b8ef-4765-9bbf-887180ac277b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:43.454241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.454177    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2" pod "77d7f8dc-2240-416a-ad87-59536dd9bf70" (UID: "77d7f8dc-2240-416a-ad87-59536dd9bf70"). InnerVolumeSpecName "kube-api-access-2vvr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:43.454555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.454531    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca" pod "77d7f8dc-2240-416a-ad87-59536dd9bf70" (UID: "77d7f8dc-2240-416a-ad87-59536dd9bf70"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:43.526761 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.526731    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:29:43.528341 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:43.528306    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c4f614_8b00_4946_8f5e_2df9d67b6d39.slice/crio-a6a0bc2f467a5da2f56e18e2330a16acacb77bc678ed2c90a9dbe22ca85adca5 WatchSource:0}: Error finding container a6a0bc2f467a5da2f56e18e2330a16acacb77bc678ed2c90a9dbe22ca85adca5: Status 404 returned error can't find the container with id a6a0bc2f467a5da2f56e18e2330a16acacb77bc678ed2c90a9dbe22ca85adca5
Oct 30 19:29:43.531191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.531153    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:43.531263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.531225    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:43.531263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.531255    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:43.550497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.550475    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:43.550497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.550498    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5d93145-b8ef-4765-9bbf-887180ac277b-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:43.550616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.550509    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vvr2\" (UniqueName: \"kubernetes.io/projected/77d7f8dc-2240-416a-ad87-59536dd9bf70-kube-api-access-2vvr2\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:43.550616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.550518    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/77d7f8dc-2240-416a-ad87-59536dd9bf70-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:43.550616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:43.550528    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72j8q\" (UniqueName: \"kubernetes.io/projected/e5d93145-b8ef-4765-9bbf-887180ac277b-kube-api-access-72j8q\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:44.219068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.219033    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb" event={"ID":"e5d93145-b8ef-4765-9bbf-887180ac277b","Type":"ContainerDied","Data":"1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004"}
Oct 30 19:29:44.219068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.219069    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"
Oct 30 19:29:44.219324 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.219090    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c90987e5604233e2a1178cc235ec9903053847263ece5b68e740e242b1c9004"
Oct 30 19:29:44.221085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.220825    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" event={"ID":"36c4f614-8b00-4946-8f5e-2df9d67b6d39","Type":"ContainerStarted","Data":"e76f880b7609eb02456f1477d195680e9d5d4abf149ab4ffec9b7e2c8f94cbf1"}
Oct 30 19:29:44.221085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.220867    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" event={"ID":"36c4f614-8b00-4946-8f5e-2df9d67b6d39","Type":"ContainerStarted","Data":"a6a0bc2f467a5da2f56e18e2330a16acacb77bc678ed2c90a9dbe22ca85adca5"}
Oct 30 19:29:44.222416 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.222392    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"
Oct 30 19:29:44.222512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.222427    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn" event={"ID":"77d7f8dc-2240-416a-ad87-59536dd9bf70","Type":"ContainerDied","Data":"146078e177b1601ad067e6870988dd31aa319e7321ab1f14a62f75ab5a800fb2"}
Oct 30 19:29:44.222512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.222472    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="146078e177b1601ad067e6870988dd31aa319e7321ab1f14a62f75ab5a800fb2"
Oct 30 19:29:44.248052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:44.248011    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podStartSLOduration=2.247994647 podStartE2EDuration="2.247994647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:44.246358343 +0000 UTC m=+649.215999178" watchObservedRunningTime="2026-10-30 19:29:44.247994647 +0000 UTC m=+649.217635483"
Oct 30 19:29:45.220956 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:45.220921    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:45.222424 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:45.222398    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:29:45.222823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:45.222456    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 19:29:46.221218 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:46.221187    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:29:46.221381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:46.221247    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 19:29:47.220781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:47.220745    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:29:47.221244 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:47.220810    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 19:29:48.221336 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.221300    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:29:48.221744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.221364    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 19:29:48.427776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.427744    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"]
Oct 30 19:29:48.428275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428260    5033 cpu_manager.go:414] "containerMap: removing container" podUID="77d7f8dc-2240-416a-ad87-59536dd9bf70" containerName="revoke-keys"
Oct 30 19:29:48.428324 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428276    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77d7f8dc-2240-416a-ad87-59536dd9bf70" containerName="revoke-keys"
Oct 30 19:29:48.428324 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428287    5033 cpu_manager.go:414] "containerMap: removing container" podUID="e5d93145-b8ef-4765-9bbf-887180ac277b" containerName="revoke-keys"
Oct 30 19:29:48.428324 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428292    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5d93145-b8ef-4765-9bbf-887180ac277b" containerName="revoke-keys"
Oct 30 19:29:48.428411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428392    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="77d7f8dc-2240-416a-ad87-59536dd9bf70" containerName="revoke-keys"
Oct 30 19:29:48.428411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.428399    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5d93145-b8ef-4765-9bbf-887180ac277b" containerName="revoke-keys"
Oct 30 19:29:48.439552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.432927    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.440909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.440883    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:29:48.441936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.441911    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:29:48.457220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.448918    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"]
Oct 30 19:29:48.536575 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.536501    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.536764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.536619    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfx7x\" (UniqueName: \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.536764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.536686    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.637910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.637870    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.638081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.637987    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfx7x\" (UniqueName: \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.638081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.638050    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.638922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.638885    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.646263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.642873    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.664701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.656761    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfx7x\" (UniqueName: \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\") pod \"maas-api-revoke-sub-acf6e151-2hxw9\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:48.787747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:48.786895    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:49.256689 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:49.255116    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:29:49.256689 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:49.255190    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 19:29:49.351838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:49.341361    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"]
Oct 30 19:29:50.336718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:50.334367    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9" event={"ID":"8c4512b7-3617-4636-b82d-1fe489c9df4f","Type":"ContainerStarted","Data":"fc574fa73a96db91b5fdaf3b1bb99b5a5296f5e0b5957f96bcd52f4024d6d3f5"}
Oct 30 19:29:50.336718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:50.334434    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9" event={"ID":"8c4512b7-3617-4636-b82d-1fe489c9df4f","Type":"ContainerStarted","Data":"e6b425d0a46ddb8c8010d9947405ede3bae46f1ba050274183d31aa8ebd66bad"}
Oct 30 19:29:50.358696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:50.357196    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9" podStartSLOduration=2.357179572 podStartE2EDuration="2.357179572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:50.352633278 +0000 UTC m=+655.322274118" watchObservedRunningTime="2026-10-30 19:29:50.357179572 +0000 UTC m=+655.326820407"
Oct 30 19:29:51.095773 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:51.095739    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:51.096517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:51.096495    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:51.097279 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:51.097256    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:29:51.341869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:51.341840    5033 generic.go:356] "Generic (PLEG): container finished" podID="8c4512b7-3617-4636-b82d-1fe489c9df4f" containerID="fc574fa73a96db91b5fdaf3b1bb99b5a5296f5e0b5957f96bcd52f4024d6d3f5" exitCode=0
Oct 30 19:29:51.342233 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:51.341925    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9" event={"ID":"8c4512b7-3617-4636-b82d-1fe489c9df4f","Type":"ContainerDied","Data":"fc574fa73a96db91b5fdaf3b1bb99b5a5296f5e0b5957f96bcd52f4024d6d3f5"}
Oct 30 19:29:52.526274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.526250    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:52.636762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.636670    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:29:52.637455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.637416    5033 cpu_manager.go:414] "containerMap: removing container" podUID="8c4512b7-3617-4636-b82d-1fe489c9df4f" containerName="revoke-keys"
Oct 30 19:29:52.637455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.637440    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c4512b7-3617-4636-b82d-1fe489c9df4f" containerName="revoke-keys"
Oct 30 19:29:52.637667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.637636    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c4512b7-3617-4636-b82d-1fe489c9df4f" containerName="revoke-keys"
Oct 30 19:29:52.641207 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.641185    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.654262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.654233    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"default-dockercfg-7qwvv\""
Oct 30 19:29:52.661755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.661731    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:29:52.662517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.662496    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"openshift-service-ca.crt\""
Oct 30 19:29:52.662803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.662784    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"distinct-w5-f7f075-kserve-self-signed-certs\""
Oct 30 19:29:52.664716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.664018    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"kube-root-ca.crt\""
Oct 30 19:29:52.706207 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.706181    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\") pod \"8c4512b7-3617-4636-b82d-1fe489c9df4f\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") "
Oct 30 19:29:52.706340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.706259    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\") pod \"8c4512b7-3617-4636-b82d-1fe489c9df4f\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") "
Oct 30 19:29:52.706475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.706449    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\" (UniqueName: \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\") pod \"8c4512b7-3617-4636-b82d-1fe489c9df4f\" (UID: \"8c4512b7-3617-4636-b82d-1fe489c9df4f\") "
Oct 30 19:29:52.706749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.706719    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca" pod "8c4512b7-3617-4636-b82d-1fe489c9df4f" (UID: "8c4512b7-3617-4636-b82d-1fe489c9df4f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:52.707190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.707165    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.708651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.708627    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x" pod "8c4512b7-3617-4636-b82d-1fe489c9df4f" (UID: "8c4512b7-3617-4636-b82d-1fe489c9df4f"). InnerVolumeSpecName "kube-api-access-xfx7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:52.709200 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.709174    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca" pod "8c4512b7-3617-4636-b82d-1fe489c9df4f" (UID: "8c4512b7-3617-4636-b82d-1fe489c9df4f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809332    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809419    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmfn8\" (UniqueName: \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809454    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809497    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809601    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809714    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809775    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c4512b7-3617-4636-b82d-1fe489c9df4f-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.815708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.809792    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfx7x\" (UniqueName: \"kubernetes.io/projected/8c4512b7-3617-4636-b82d-1fe489c9df4f-kube-api-access-xfx7x\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.911297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911209    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911321    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911403    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911443    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911617 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911504    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmfn8\" (UniqueName: \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911617 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911537    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911802    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.911993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.911947    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.912108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.912067    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.914377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.914352    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.914746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.914716    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.928452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.928425    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmfn8\" (UniqueName: \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\") pod \"distinct-w5-f7f075-kserve-766cc8fbd9-hk59w\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:52.928905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.928884    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"]
Oct 30 19:29:52.934877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.934859    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:52.941566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.941469    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"]
Oct 30 19:29:52.981397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:52.981360    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:53.012373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.012346    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.012480 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.012450    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.012541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.012510    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99tfj\" (UniqueName: \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.113630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.113592    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.113810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.113707    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.113810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.113774    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99tfj\" (UniqueName: \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.114753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.114725    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.117103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.117081    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.124374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.124347    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99tfj\" (UniqueName: \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\") pod \"maas-api-revoke-sub-1c6170d1-6hhmf\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.263167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.263144    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:53.270872 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.270845    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:29:53.272280 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:53.272246    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc01c0ecd_311a_4c1b_8d8a_af3e8debaef5.slice/crio-d8846157434df8b594321971f4da960f05891f4a1de9e778aeecded588b6b76e WatchSource:0}: Error finding container d8846157434df8b594321971f4da960f05891f4a1de9e778aeecded588b6b76e: Status 404 returned error can't find the container with id d8846157434df8b594321971f4da960f05891f4a1de9e778aeecded588b6b76e
Oct 30 19:29:53.365015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.362416    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" event={"ID":"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5","Type":"ContainerStarted","Data":"d8846157434df8b594321971f4da960f05891f4a1de9e778aeecded588b6b76e"}
Oct 30 19:29:53.365015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.364557    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9" event={"ID":"8c4512b7-3617-4636-b82d-1fe489c9df4f","Type":"ContainerDied","Data":"e6b425d0a46ddb8c8010d9947405ede3bae46f1ba050274183d31aa8ebd66bad"}
Oct 30 19:29:53.365015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.364611    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6b425d0a46ddb8c8010d9947405ede3bae46f1ba050274183d31aa8ebd66bad"
Oct 30 19:29:53.365015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.364698    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"
Oct 30 19:29:53.566706 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:53.566662    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"]
Oct 30 19:29:53.571552 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:29:53.571524    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb479af8d_7727_4a7d_a1e7_2fa92f405a44.slice/crio-140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522 WatchSource:0}: Error finding container 140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522: Status 404 returned error can't find the container with id 140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522
Oct 30 19:29:54.377736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:54.374486    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" event={"ID":"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5","Type":"ContainerStarted","Data":"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"}
Oct 30 19:29:54.377736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:54.376838    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf" event={"ID":"b479af8d-7727-4a7d-a1e7-2fa92f405a44","Type":"ContainerStarted","Data":"d5f96c38ea7240a6cb9cfdfebb3060681f2fdd39ad48706503f5ca8eb3f96248"}
Oct 30 19:29:54.377736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:54.376896    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf" event={"ID":"b479af8d-7727-4a7d-a1e7-2fa92f405a44","Type":"ContainerStarted","Data":"140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522"}
Oct 30 19:29:54.395294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:54.395249    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podStartSLOduration=2.39523668 podStartE2EDuration="2.39523668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:54.392837743 +0000 UTC m=+659.362478579" watchObservedRunningTime="2026-10-30 19:29:54.39523668 +0000 UTC m=+659.364877543"
Oct 30 19:29:54.981576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:54.981540    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:29:55.392142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.392059    5033 generic.go:356] "Generic (PLEG): container finished" podID="b479af8d-7727-4a7d-a1e7-2fa92f405a44" containerID="d5f96c38ea7240a6cb9cfdfebb3060681f2fdd39ad48706503f5ca8eb3f96248" exitCode=0
Oct 30 19:29:55.392356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.392323    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf" event={"ID":"b479af8d-7727-4a7d-a1e7-2fa92f405a44","Type":"ContainerDied","Data":"d5f96c38ea7240a6cb9cfdfebb3060681f2fdd39ad48706503f5ca8eb3f96248"}
Oct 30 19:29:55.589211 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.589187    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:55.649205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.649125    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\") pod \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") "
Oct 30 19:29:55.649551 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.649242    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\" (UniqueName: \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\") pod \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") "
Oct 30 19:29:55.649551 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.649315    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\") pod \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\" (UID: \"b479af8d-7727-4a7d-a1e7-2fa92f405a44\") "
Oct 30 19:29:55.649772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.649743    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca" pod "b479af8d-7727-4a7d-a1e7-2fa92f405a44" (UID: "b479af8d-7727-4a7d-a1e7-2fa92f405a44"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:55.649978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.649957    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:55.654054 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.654026    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca" pod "b479af8d-7727-4a7d-a1e7-2fa92f405a44" (UID: "b479af8d-7727-4a7d-a1e7-2fa92f405a44"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:55.654517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.654490    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj" pod "b479af8d-7727-4a7d-a1e7-2fa92f405a44" (UID: "b479af8d-7727-4a7d-a1e7-2fa92f405a44"). InnerVolumeSpecName "kube-api-access-99tfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:55.750992 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.750965    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b479af8d-7727-4a7d-a1e7-2fa92f405a44-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:55.750992 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:55.750993    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99tfj\" (UniqueName: \"kubernetes.io/projected/b479af8d-7727-4a7d-a1e7-2fa92f405a44-kube-api-access-99tfj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.401637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:56.401603    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"
Oct 30 19:29:56.401637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:56.401613    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf" event={"ID":"b479af8d-7727-4a7d-a1e7-2fa92f405a44","Type":"ContainerDied","Data":"140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522"}
Oct 30 19:29:56.402106 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:56.401671    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="140be0feee226a580ca4228ac0755472fa8ca96aaaf2ce0b16a22ee4ff73e522"
Oct 30 19:29:58.159704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.150534    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:29:58.159704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.151546    5033 cpu_manager.go:414] "containerMap: removing container" podUID="b479af8d-7727-4a7d-a1e7-2fa92f405a44" containerName="revoke-keys"
Oct 30 19:29:58.159704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.151567    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b479af8d-7727-4a7d-a1e7-2fa92f405a44" containerName="revoke-keys"
Oct 30 19:29:58.159704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.151922    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="b479af8d-7727-4a7d-a1e7-2fa92f405a44" containerName="revoke-keys"
Oct 30 19:29:58.188722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.181960    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.359701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.357164    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-639c15-openshift-default-dockercfg-vrz5l\""
Oct 30 19:29:58.440755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.424403    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.440755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.424459    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.440755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.424514    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.464754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.441153    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.464754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.441214    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.464754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.441347    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.464754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.441401    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjdmr\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.464754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.441506    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.466330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.465518    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617034    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617118    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617187    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617383    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617415    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617498    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617530    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617641    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.626072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.617873    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjdmr\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.684792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.684756    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:29:58.713999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.712859    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.713999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.713567    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.866555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.837233    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.866555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.837542    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.866555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.837845    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:58.992633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:58.992548    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:59.033204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:59.033161    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:59.034831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:59.034791    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjdmr\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:59.082523 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:59.082487    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:29:59.287904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:29:59.287822    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:30:00.030720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.019071    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:30:00.035751 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:00.035614    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec69a62c_7143_4ef9_849e_15d6b5529ea2.slice/crio-dfd077751cb3f1b34617b82c2017cef06fee028b6611d1a3682815c6ceb69a17 WatchSource:0}: Error finding container dfd077751cb3f1b34617b82c2017cef06fee028b6611d1a3682815c6ceb69a17: Status 404 returned error can't find the container with id dfd077751cb3f1b34617b82c2017cef06fee028b6611d1a3682815c6ceb69a17
Oct 30 19:30:00.038805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.038761    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:00.039007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.038981    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:00.039173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.039152    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:00.189701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.188901    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:00.210602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.210574    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.215115 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.215080    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:00.219757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.219546    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:30:00.221698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.221638    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.230003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.229981    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:00.280475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.279655    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9m9d\" (UniqueName: \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.280475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.279729    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.280475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.279904    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7h7q\" (UniqueName: \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.280475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.279998    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.314152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.314082    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.315178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.313991    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:00.315178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.314794    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:00.383698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.383491    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:00.383698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.383537    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:00.383698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.383571    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:00.383698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.383659    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390252    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390345    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7h7q\" (UniqueName: \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390413    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390459    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trkn4\" (UniqueName: \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390557    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9m9d\" (UniqueName: \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.392300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.390604    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.392743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.392372    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.395127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.395102    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.474381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.474326    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7h7q\" (UniqueName: \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\") pod \"maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.504726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.501785    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.504726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.501939    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trkn4\" (UniqueName: \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.504726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.503433    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.506915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.506751    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9m9d\" (UniqueName: \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\") pod \"maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.546016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.545948    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trkn4\" (UniqueName: \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\") pod \"maas-api-key-cleanup-29889810-qshk8\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.597954 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.597879    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:00.607697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.603283    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7kzp\" (UniqueName: \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.607697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.603501    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.607697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.603706    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:00.647742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.647718    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:00.809286 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.809248    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7kzp\" (UniqueName: \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.816698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.812966    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.834698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.817699    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:00.975711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:00.975667    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7kzp\" (UniqueName: \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\") pod \"maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:01.125714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.070552    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" event={"ID":"ec69a62c-7143-4ef9-849e-15d6b5529ea2","Type":"ContainerStarted","Data":"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"}
Oct 30 19:30:01.125714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.070620    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" event={"ID":"ec69a62c-7143-4ef9-849e-15d6b5529ea2","Type":"ContainerStarted","Data":"dfd077751cb3f1b34617b82c2017cef06fee028b6611d1a3682815c6ceb69a17"}
Oct 30 19:30:01.143840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.143777    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podStartSLOduration=3.143759224 podStartE2EDuration="3.143759224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:01.142349377 +0000 UTC m=+666.111990214" watchObservedRunningTime="2026-10-30 19:30:01.143759224 +0000 UTC m=+666.113400061"
Oct 30 19:30:01.173314 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.173068    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:01.274403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.274063    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:30:01.274403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.274123    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:01.289696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.289500    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:30:01.308826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.290805    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:01.308826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.290849    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:01.581863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:01.579382    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:02.224896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.223390    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:02.407699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.407056    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:02.407699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.407152    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:02.463235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.463203    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:02.469406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.469381    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:02.625037 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.624988    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:02.626862 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.626825    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:02.635263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.635144    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:02.680066 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.680028    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerStarted","Data":"dc1a0dca63cf3d14a68f7f5b9d562ad36937987bd64336391f6a4a818eae2dcc"}
Oct 30 19:30:02.732175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.732141    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerStarted","Data":"fa7beaf99e6e48ad6144727fbcc66247afa85f7159f69b388ee6416e9c108702"}
Oct 30 19:30:02.921157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.917774    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:30:02.986931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:02.985506    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:30:03.138701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.133262    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:30:03.290473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.290106    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:03.290473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.290156    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:03.741485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.738743    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerStarted","Data":"77861e4a86c743e258bba4ed9fc4b79b831d8b46b6d8db368e938b095e06be82"}
Oct 30 19:30:03.741485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.738812    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerStarted","Data":"4d9bac53079a426a2f92eddafc6d6375396ae98655d4498c6e3c579ffb02e028"}
Oct 30 19:30:03.743473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.743438    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerStarted","Data":"dee24147ab3dfe09a6ac71cec803a7a236bc3f306d812a2933777390956d89b5"}
Oct 30 19:30:03.745782 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.745757    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerStarted","Data":"0480146ae70c60f9deff17961a2971c03d7a683b986cb51bc350e541650753f8"}
Oct 30 19:30:03.745898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.745806    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerStarted","Data":"d4c41a686b0b56716a88f88b0e39531deb516e40404689c5df809a35582f16ab"}
Oct 30 19:30:03.747725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.747699    5033 generic.go:356] "Generic (PLEG): container finished" podID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerID="ad21659feffe2cd91f9a3bb100df2b0842079c7e7e54e87357993bec4c668265" exitCode=60
Oct 30 19:30:03.748831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.748813    5033 scope.go:122] "RemoveContainer" containerID="ad21659feffe2cd91f9a3bb100df2b0842079c7e7e54e87357993bec4c668265"
Oct 30 19:30:03.749156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.749136    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerDied","Data":"ad21659feffe2cd91f9a3bb100df2b0842079c7e7e54e87357993bec4c668265"}
Oct 30 19:30:03.806837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.783651    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" podStartSLOduration=3.783632204 podStartE2EDuration="3.783632204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.76209574 +0000 UTC m=+668.731736576" watchObservedRunningTime="2026-10-30 19:30:03.783632204 +0000 UTC m=+668.753273041"
Oct 30 19:30:03.806837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.806769    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:30:03.829611 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.829337    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" podStartSLOduration=3.829318219 podStartE2EDuration="3.829318219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.826741748 +0000 UTC m=+668.796382584" watchObservedRunningTime="2026-10-30 19:30:03.829318219 +0000 UTC m=+668.798959055"
Oct 30 19:30:03.847965 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:03.846943    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" podStartSLOduration=3.846925486 podStartE2EDuration="3.846925486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.844310742 +0000 UTC m=+668.813951578" watchObservedRunningTime="2026-10-30 19:30:03.846925486 +0000 UTC m=+668.816566322"
Oct 30 19:30:04.291953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.291123    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:04.291953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.291174    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:04.786807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.786746    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerStarted","Data":"50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"}
Oct 30 19:30:04.790171 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.790142    5033 generic.go:356] "Generic (PLEG): container finished" podID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerID="77861e4a86c743e258bba4ed9fc4b79b831d8b46b6d8db368e938b095e06be82" exitCode=60
Oct 30 19:30:04.790596 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.790250    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerDied","Data":"77861e4a86c743e258bba4ed9fc4b79b831d8b46b6d8db368e938b095e06be82"}
Oct 30 19:30:04.791504 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.790716    5033 scope.go:122] "RemoveContainer" containerID="77861e4a86c743e258bba4ed9fc4b79b831d8b46b6d8db368e938b095e06be82"
Oct 30 19:30:04.796472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.794859    5033 generic.go:356] "Generic (PLEG): container finished" podID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerID="dee24147ab3dfe09a6ac71cec803a7a236bc3f306d812a2933777390956d89b5" exitCode=60
Oct 30 19:30:04.796472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.795234    5033 scope.go:122] "RemoveContainer" containerID="dee24147ab3dfe09a6ac71cec803a7a236bc3f306d812a2933777390956d89b5"
Oct 30 19:30:04.796472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.795513    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerDied","Data":"dee24147ab3dfe09a6ac71cec803a7a236bc3f306d812a2933777390956d89b5"}
Oct 30 19:30:04.803531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.802337    5033 generic.go:356] "Generic (PLEG): container finished" podID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerID="0480146ae70c60f9deff17961a2971c03d7a683b986cb51bc350e541650753f8" exitCode=60
Oct 30 19:30:04.803531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.803203    5033 scope.go:122] "RemoveContainer" containerID="0480146ae70c60f9deff17961a2971c03d7a683b986cb51bc350e541650753f8"
Oct 30 19:30:04.803531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.803466    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerDied","Data":"0480146ae70c60f9deff17961a2971c03d7a683b986cb51bc350e541650753f8"}
Oct 30 19:30:04.829156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:04.829106    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" podStartSLOduration=4.829091292 podStartE2EDuration="4.829091292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:04.810169935 +0000 UTC m=+669.779810771" watchObservedRunningTime="2026-10-30 19:30:04.829091292 +0000 UTC m=+669.798732128"
Oct 30 19:30:05.105185 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:05.105140    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:30:05.290069 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.290037    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:05.290178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.290090    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:05.297865 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.297839    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"]
Oct 30 19:30:05.310003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.309585    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.310003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.309606    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"]
Oct 30 19:30:05.318711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.316937    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:30:05.423100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.423013    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.423100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.423058    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rpff\" (UniqueName: \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.423100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.423091    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.524212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.524181    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.524391 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.524222    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rpff\" (UniqueName: \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.524391 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.524259    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.525058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.525028    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.529089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.529060    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.541870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.541843    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rpff\" (UniqueName: \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\") pod \"maas-api-revoke-sub-8cb6322e-hgvt6\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.632507 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.632468    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:05.815476 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.815440    5033 generic.go:356] "Generic (PLEG): container finished" podID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerID="50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d" exitCode=60
Oct 30 19:30:05.815979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.815554    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerDied","Data":"50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"}
Oct 30 19:30:05.815979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.815616    5033 scope.go:122] "RemoveContainer" containerID="ad21659feffe2cd91f9a3bb100df2b0842079c7e7e54e87357993bec4c668265"
Oct 30 19:30:05.816983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.816110    5033 scope.go:122] "RemoveContainer" containerID="50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"
Oct 30 19:30:05.816983 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:05.816478    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv_odh-ai-gateway-infra(5bc93ffe-e598-4fd4-a5bc-f76819afc7a0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0"
Oct 30 19:30:05.818886 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.818856    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerStarted","Data":"c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"}
Oct 30 19:30:05.822368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.822329    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerStarted","Data":"a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"}
Oct 30 19:30:05.825364 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.825339    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerStarted","Data":"b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"}
Oct 30 19:30:05.954544 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:05.950713    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"]
Oct 30 19:30:05.954544 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:05.951193    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a6c99b_ab1a_483c_861a_7049a68b0a07.slice/crio-2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8 WatchSource:0}: Error finding container 2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8: Status 404 returned error can't find the container with id 2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8
Oct 30 19:30:06.289241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.289151    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:06.289241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.289201    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:06.837205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.837163    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6" event={"ID":"32a6c99b-ab1a-483c-861a-7049a68b0a07","Type":"ContainerStarted","Data":"0cc83cf141f43fce39fdc271cff310e3586188a0866600894e7886450011876c"}
Oct 30 19:30:06.837728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.837234    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6" event={"ID":"32a6c99b-ab1a-483c-861a-7049a68b0a07","Type":"ContainerStarted","Data":"2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8"}
Oct 30 19:30:06.840202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.840176    5033 scope.go:122] "RemoveContainer" containerID="50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"
Oct 30 19:30:06.840506 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:06.840479    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv_odh-ai-gateway-infra(5bc93ffe-e598-4fd4-a5bc-f76819afc7a0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0"
Oct 30 19:30:06.842440 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.842360    5033 generic.go:356] "Generic (PLEG): container finished" podID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerID="c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19" exitCode=60
Oct 30 19:30:06.842563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.842519    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerDied","Data":"c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"}
Oct 30 19:30:06.842627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.842569    5033 scope.go:122] "RemoveContainer" containerID="77861e4a86c743e258bba4ed9fc4b79b831d8b46b6d8db368e938b095e06be82"
Oct 30 19:30:06.842738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.842715    5033 scope.go:122] "RemoveContainer" containerID="c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"
Oct 30 19:30:06.844619 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:06.844593    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d_odh-ai-gateway-infra(3576df6c-35f3-4a36-93be-9756930eb7ca)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca"
Oct 30 19:30:06.845512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.845474    5033 generic.go:356] "Generic (PLEG): container finished" podID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerID="a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e" exitCode=60
Oct 30 19:30:06.845630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.845556    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerDied","Data":"a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"}
Oct 30 19:30:06.845813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.845782    5033 scope.go:122] "RemoveContainer" containerID="a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"
Oct 30 19:30:06.846015 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:06.845994    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-qshk8_odh-ai-gateway-infra(4961f1da-df1d-4b55-8633-50f7ba990f52)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52"
Oct 30 19:30:06.848095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.847862    5033 generic.go:356] "Generic (PLEG): container finished" podID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerID="b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a" exitCode=60
Oct 30 19:30:06.848095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.847980    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerDied","Data":"b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"}
Oct 30 19:30:06.870235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.848601    5033 scope.go:122] "RemoveContainer" containerID="b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"
Oct 30 19:30:06.870235 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:06.848836    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d_odh-ai-gateway-infra(cba930e3-3e7c-41f1-bbc0-733d44aa5ccc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc"
Oct 30 19:30:06.879710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.879076    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6" podStartSLOduration=1.879056511 podStartE2EDuration="1.879056511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:06.877136275 +0000 UTC m=+671.846777113" watchObservedRunningTime="2026-10-30 19:30:06.879056511 +0000 UTC m=+671.848697347"
Oct 30 19:30:06.978727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:06.978010    5033 scope.go:122] "RemoveContainer" containerID="dee24147ab3dfe09a6ac71cec803a7a236bc3f306d812a2933777390956d89b5"
Oct 30 19:30:07.127011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.126268    5033 scope.go:122] "RemoveContainer" containerID="0480146ae70c60f9deff17961a2971c03d7a683b986cb51bc350e541650753f8"
Oct 30 19:30:07.289128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.289093    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:30:07.289294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.289157    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 19:30:07.858502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.858459    5033 generic.go:356] "Generic (PLEG): container finished" podID="32a6c99b-ab1a-483c-861a-7049a68b0a07" containerID="0cc83cf141f43fce39fdc271cff310e3586188a0866600894e7886450011876c" exitCode=0
Oct 30 19:30:07.859107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.858592    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6" event={"ID":"32a6c99b-ab1a-483c-861a-7049a68b0a07","Type":"ContainerDied","Data":"0cc83cf141f43fce39fdc271cff310e3586188a0866600894e7886450011876c"}
Oct 30 19:30:07.861533 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.861503    5033 scope.go:122] "RemoveContainer" containerID="c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"
Oct 30 19:30:07.861918 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:07.861890    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d_odh-ai-gateway-infra(3576df6c-35f3-4a36-93be-9756930eb7ca)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca"
Oct 30 19:30:07.864506 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.864484    5033 scope.go:122] "RemoveContainer" containerID="a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"
Oct 30 19:30:07.864844 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:07.864811    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-qshk8_odh-ai-gateway-infra(4961f1da-df1d-4b55-8633-50f7ba990f52)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52"
Oct 30 19:30:07.866261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:07.866241    5033 scope.go:122] "RemoveContainer" containerID="b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"
Oct 30 19:30:07.866527 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:07.866506    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d_odh-ai-gateway-infra(cba930e3-3e7c-41f1-bbc0-733d44aa5ccc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc"
Oct 30 19:30:08.298400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:08.298371    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:30:08.298742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:08.298719    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:30:08.299582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:08.299561    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:30:09.038041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.038010    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:09.117954 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.117905    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\" (UniqueName: \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\") pod \"32a6c99b-ab1a-483c-861a-7049a68b0a07\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") "
Oct 30 19:30:09.118116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.118050    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\") pod \"32a6c99b-ab1a-483c-861a-7049a68b0a07\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") "
Oct 30 19:30:09.118116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.118106    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\") pod \"32a6c99b-ab1a-483c-861a-7049a68b0a07\" (UID: \"32a6c99b-ab1a-483c-861a-7049a68b0a07\") "
Oct 30 19:30:09.118753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.118721    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca" pod "32a6c99b-ab1a-483c-861a-7049a68b0a07" (UID: "32a6c99b-ab1a-483c-861a-7049a68b0a07"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:09.120576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.120540    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff" pod "32a6c99b-ab1a-483c-861a-7049a68b0a07" (UID: "32a6c99b-ab1a-483c-861a-7049a68b0a07"). InnerVolumeSpecName "kube-api-access-6rpff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:09.120712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.120627    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca" pod "32a6c99b-ab1a-483c-861a-7049a68b0a07" (UID: "32a6c99b-ab1a-483c-861a-7049a68b0a07"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:09.220323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.220288    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:09.220323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.220318    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a6c99b-ab1a-483c-861a-7049a68b0a07-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:09.220503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.220332    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rpff\" (UniqueName: \"kubernetes.io/projected/32a6c99b-ab1a-483c-861a-7049a68b0a07-kube-api-access-6rpff\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:09.883229 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.883197    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"
Oct 30 19:30:09.883494 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.883454    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6" event={"ID":"32a6c99b-ab1a-483c-861a-7049a68b0a07","Type":"ContainerDied","Data":"2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8"}
Oct 30 19:30:09.883665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:09.883645    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2281845c8307d889f83e7084b905670a4f44f15e03c2591134bacbeb9a1562d8"
Oct 30 19:30:13.407716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.405141    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:30:13.407716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.405998    5033 cpu_manager.go:414] "containerMap: removing container" podUID="32a6c99b-ab1a-483c-861a-7049a68b0a07" containerName="revoke-keys"
Oct 30 19:30:13.407716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.406014    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32a6c99b-ab1a-483c-861a-7049a68b0a07" containerName="revoke-keys"
Oct 30 19:30:13.407716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.406163    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="32a6c99b-ab1a-483c-861a-7049a68b0a07" containerName="revoke-keys"
Oct 30 19:30:13.428846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.411085    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.453762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.451644    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.503737    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.503884    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.503923    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.503967    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.504040    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.504090    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.504126    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2fp5\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.504177    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.536696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.504208    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.605582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.605543    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.606238 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.606147    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.606626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.606589    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.606730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.606656    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.606787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.606726    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2fp5\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.607192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.607151    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.608350 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.607213    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.608459 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.607952    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.608516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.608320    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.608516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.608419    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.608910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.608884    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.609323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.609300    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.609837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.609814    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.609988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.609249    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.612833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.612806    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.632435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.632388    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.696958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.672363    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2fp5\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.731013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.727050    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:13.792035 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:13.788616    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:14.752463 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:14.752425    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:30:14.846142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:14.846098    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:14.846400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:14.846369    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:14.846616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:14.846577    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:14.998394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:14.998363    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:30:15.109966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.109866    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" event={"ID":"359196ec-6ea5-4c10-ac93-2cb03f88a4ee","Type":"ContainerStarted","Data":"9d71468052e180a0b85299328dbb6e949512cda3828cee24bab97d24f06ec6ff"}
Oct 30 19:30:15.110323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.110259    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.155711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.142339    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"distinct-w0-171808-kserve-self-signed-certs\""
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250087    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250172    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250262    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250321    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250353    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l65d9\" (UniqueName: \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.264474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.250493    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.327757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.327725    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:30:15.353425 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.353388    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.372556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.372480    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.373131 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.373103    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l65d9\" (UniqueName: \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.373974 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.373954    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.385797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.354565    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.385956 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.373028    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.386369 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.386349    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.387179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.387161    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.388395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.388373    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.437909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.430361    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.437909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.434999    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.486700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.486539    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l65d9\" (UniqueName: \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\") pod \"distinct-w0-171808-kserve-695b79d8b4-wz9ln\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") " pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.641576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.641496    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:15.665973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.665931    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:15.715946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.715900    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:15.719189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.718826    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:15.754866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.754820    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:15.948065 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:15.947972    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.311414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.304254    5033 scope.go:122] "RemoveContainer" containerID="50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"
Oct 30 19:30:16.736976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.718221    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.736976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.720074    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.736976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.720156    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:30:16.736976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.722551    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:30:16.744697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.737972    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.745881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.745849    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:30:16.746115 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.746090    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:30:16.772990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.766477    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:30:16.837701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.794804    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:30:16.850709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.800642    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.850709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.848168    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:30:16.850709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.849753    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.899901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.894482    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" event={"ID":"359196ec-6ea5-4c10-ac93-2cb03f88a4ee","Type":"ContainerStarted","Data":"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"}
Oct 30 19:30:16.973704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.943777    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.973704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.969785    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.973704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:16.970224    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:30:17.168834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.168800    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:30:17.194834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.194759    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podStartSLOduration=4.194739166 podStartE2EDuration="4.194739166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:17.101842036 +0000 UTC m=+682.071482876" watchObservedRunningTime="2026-10-30 19:30:17.194739166 +0000 UTC m=+682.164380003"
Oct 30 19:30:17.360815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.353189    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:30:17.626181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.625429    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:30:17.731696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.728790    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:30:17.796785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.792168    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:17.808852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.807809    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:17.808852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.807870    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:17.824687 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:17.811765    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:18.106615 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.106570    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"]
Oct 30 19:30:18.133430 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.131895    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.141663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.141605    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:30:18.148567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.148499    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerStarted","Data":"ee86b12e69ec628df5d87bc870cb4a3589edd76567abeeb7a88ff0ce7f463761"}
Oct 30 19:30:18.197472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.196401    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" event={"ID":"a624dad3-3364-4c6c-a18c-382928c0e89c","Type":"ContainerStarted","Data":"d4769b3a42a948588fd4093260a139b46d14f2dc7898c706e59b7c45ed341b7d"}
Oct 30 19:30:18.217971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.202382    5033 scope.go:122] "RemoveContainer" containerID="c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"
Oct 30 19:30:18.217971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.207833    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"]
Oct 30 19:30:18.217971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.217609    5033 scope.go:122] "RemoveContainer" containerID="b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"
Oct 30 19:30:18.271263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.271195    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:30:18.823396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.823323    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:18.824511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.823958    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:18.839717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.828865    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.839717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.829038    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smwn\" (UniqueName: \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.839717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.829217    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.938012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.937968    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9smwn\" (UniqueName: \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.938291 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.938273    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.938688 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.938656    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.976946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.957091    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:18.987510 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:18.987477    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:19.020497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.020422    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smwn\" (UniqueName: \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\") pod \"maas-api-revoke-sub-a3d5b3a9-m5bzw\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:19.172701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.169415    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:19.233448 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.233330    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:19.241374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.241340    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerStarted","Data":"4235718ce8e2e2204df9fe61812e0aff6bd45f0d997c9120d75e4f019c2f5fd1"}
Oct 30 19:30:19.245305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.245278    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" event={"ID":"a624dad3-3364-4c6c-a18c-382928c0e89c","Type":"ContainerStarted","Data":"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"}
Oct 30 19:30:19.263964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.263933    5033 generic.go:356] "Generic (PLEG): container finished" podID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerID="ee86b12e69ec628df5d87bc870cb4a3589edd76567abeeb7a88ff0ce7f463761" exitCode=60
Oct 30 19:30:19.264353 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.264308    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerDied","Data":"ee86b12e69ec628df5d87bc870cb4a3589edd76567abeeb7a88ff0ce7f463761"}
Oct 30 19:30:19.264492 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.264374    5033 scope.go:122] "RemoveContainer" containerID="50e939ec1c5205d80108cec7eb2518464fba245bd2ae14eee144a18b3786312d"
Oct 30 19:30:19.342042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.341986    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podStartSLOduration=5.341966621 podStartE2EDuration="5.341966621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:19.339584171 +0000 UTC m=+684.309225011" watchObservedRunningTime="2026-10-30 19:30:19.341966621 +0000 UTC m=+684.311607458"
Oct 30 19:30:19.644854 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.644812    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:19.712185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.712158    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"]
Oct 30 19:30:19.801704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.797188    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:19.801704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.797241    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:19.848122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.848104    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:19.867748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.866142    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\") pod \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") "
Oct 30 19:30:19.867748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.866243    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\" (UniqueName: \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\") pod \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\" (UID: \"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0\") "
Oct 30 19:30:19.868272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.868216    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca" pod "5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" (UID: "5bc93ffe-e598-4fd4-a5bc-f76819afc7a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:19.884701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.878829    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q" pod "5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" (UID: "5bc93ffe-e598-4fd4-a5bc-f76819afc7a0"). InnerVolumeSpecName "kube-api-access-m7h7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:19.969575 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.969543    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-maas-api-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:19.969785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:19.969764    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7h7q\" (UniqueName: \"kubernetes.io/projected/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0-kube-api-access-m7h7q\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.164472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.164396    5033 scope.go:122] "RemoveContainer" containerID="a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"
Oct 30 19:30:20.316700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.314963    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:20.321916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.321883    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw" event={"ID":"5b66f84b-3a5e-450c-9e9b-ee2e3e290628","Type":"ContainerStarted","Data":"213a4b803ffdd54f39dfdff2b0a1b872bfc6646263bdf0cf5470355477fa0506"}
Oct 30 19:30:20.322105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.322086    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw" event={"ID":"5b66f84b-3a5e-450c-9e9b-ee2e3e290628","Type":"ContainerStarted","Data":"c3f4b4fe3befd08f58af7e882141d5c7e40b30da3cf143bb9f5c6775fb350c7f"}
Oct 30 19:30:20.324550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.324453    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv" event={"ID":"5bc93ffe-e598-4fd4-a5bc-f76819afc7a0","Type":"ContainerDied","Data":"fa7beaf99e6e48ad6144727fbcc66247afa85f7159f69b388ee6416e9c108702"}
Oct 30 19:30:20.324550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.324526    5033 scope.go:122] "RemoveContainer" containerID="ee86b12e69ec628df5d87bc870cb4a3589edd76567abeeb7a88ff0ce7f463761"
Oct 30 19:30:20.324757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.324652    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"
Oct 30 19:30:20.332643 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.332603    5033 generic.go:356] "Generic (PLEG): container finished" podID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerID="4235718ce8e2e2204df9fe61812e0aff6bd45f0d997c9120d75e4f019c2f5fd1" exitCode=60
Oct 30 19:30:20.332772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.332754    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerDied","Data":"4235718ce8e2e2204df9fe61812e0aff6bd45f0d997c9120d75e4f019c2f5fd1"}
Oct 30 19:30:20.336127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.336108    5033 generic.go:356] "Generic (PLEG): container finished" podID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerID="20cbb5c41e05eef3711401f2fe9ad0080a206a80ad4543448e36437ddefe4b39" exitCode=60
Oct 30 19:30:20.336224 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.336170    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerDied","Data":"20cbb5c41e05eef3711401f2fe9ad0080a206a80ad4543448e36437ddefe4b39"}
Oct 30 19:30:20.336616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.336598    5033 scope.go:122] "RemoveContainer" containerID="20cbb5c41e05eef3711401f2fe9ad0080a206a80ad4543448e36437ddefe4b39"
Oct 30 19:30:20.336895 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:20.336877    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d_odh-ai-gateway-infra(cba930e3-3e7c-41f1-bbc0-733d44aa5ccc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc"
Oct 30 19:30:20.423466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.423430    5033 scope.go:122] "RemoveContainer" containerID="c6d3affcdcfcc626b5d805c6a5850ff0ccf499cc9bd5fc9053cae315ecbfdc19"
Oct 30 19:30:20.438515 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.438488    5033 scope.go:122] "RemoveContainer" containerID="b1c40b21bf02c6d944599dde108d86d05008d554467357d63a141e10285d478a"
Oct 30 19:30:20.446752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.446711    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw" podStartSLOduration=3.446698634 podStartE2EDuration="3.446698634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:20.405898941 +0000 UTC m=+685.375539778" watchObservedRunningTime="2026-10-30 19:30:20.446698634 +0000 UTC m=+685.416339468"
Oct 30 19:30:20.522516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.522478    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:20.528802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.528773    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-9f2d57-29889810-kdmgv"]
Oct 30 19:30:20.638172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.638149    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:20.717823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.717794    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\" (UniqueName: \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\") pod \"3576df6c-35f3-4a36-93be-9756930eb7ca\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") "
Oct 30 19:30:20.717969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.717950    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\") pod \"3576df6c-35f3-4a36-93be-9756930eb7ca\" (UID: \"3576df6c-35f3-4a36-93be-9756930eb7ca\") "
Oct 30 19:30:20.718915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.718784    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca" pod "3576df6c-35f3-4a36-93be-9756930eb7ca" (UID: "3576df6c-35f3-4a36-93be-9756930eb7ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:20.721286 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.721261    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d" pod "3576df6c-35f3-4a36-93be-9756930eb7ca" (UID: "3576df6c-35f3-4a36-93be-9756930eb7ca"). InnerVolumeSpecName "kube-api-access-d9m9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:20.789581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.789549    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:20.789742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.789604    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:20.819363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.819340    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9m9d\" (UniqueName: \"kubernetes.io/projected/3576df6c-35f3-4a36-93be-9756930eb7ca-kube-api-access-d9m9d\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.819363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.819364    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3576df6c-35f3-4a36-93be-9756930eb7ca-maas-api-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.899708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.898827    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:30:20.899708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.899690    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.899708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.899713    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.900310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.899733    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.900310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.899741    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914437 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914403    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914437 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914439    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914484    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914493    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914538    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914545    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914579    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.914667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914590    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.914979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914794    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914811    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:20.914979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914826    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.914979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.914835    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:20.920947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.920917    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:30:20.921075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.921024    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:20.929857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:20.929077    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f7f075\"/\"distinct-2-w5-f7f075-kserve-self-signed-certs\""
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046034    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046104    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046157    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046201    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pscdw\" (UniqueName: \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046394    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.053709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.046437    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147392    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147446    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147490    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147523    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pscdw\" (UniqueName: \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147714    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.147746    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.148191    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.148403    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.149772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.148917    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.173700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.169751    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.173700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.170278    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.185866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.176541    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pscdw\" (UniqueName: \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\") pod \"distinct-2-w5-f7f075-kserve-64989fc678-nkrph\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") " pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.186055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.183144    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" path="/var/lib/kubelet/pods/5bc93ffe-e598-4fd4-a5bc-f76819afc7a0/volumes"
Oct 30 19:30:21.234731 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.234700    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:21.346964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.346877    5033 generic.go:356] "Generic (PLEG): container finished" podID="5b66f84b-3a5e-450c-9e9b-ee2e3e290628" containerID="213a4b803ffdd54f39dfdff2b0a1b872bfc6646263bdf0cf5470355477fa0506" exitCode=0
Oct 30 19:30:21.347143 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.346981    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw" event={"ID":"5b66f84b-3a5e-450c-9e9b-ee2e3e290628","Type":"ContainerDied","Data":"213a4b803ffdd54f39dfdff2b0a1b872bfc6646263bdf0cf5470355477fa0506"}
Oct 30 19:30:21.363696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.361023    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d" event={"ID":"3576df6c-35f3-4a36-93be-9756930eb7ca","Type":"ContainerDied","Data":"4d9bac53079a426a2f92eddafc6d6375396ae98655d4498c6e3c579ffb02e028"}
Oct 30 19:30:21.363696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.361087    5033 scope.go:122] "RemoveContainer" containerID="4235718ce8e2e2204df9fe61812e0aff6bd45f0d997c9120d75e4f019c2f5fd1"
Oct 30 19:30:21.363696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.361222    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"
Oct 30 19:30:21.411465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.411385    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerStarted","Data":"f86344838a183cce37adcb7bf23b6294759d65190567b0734820220bf2fea621"}
Oct 30 19:30:21.493766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.493736    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:21.496262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.496230    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-171808-29889810-trv7d"]
Oct 30 19:30:21.500911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.500890    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:21.695858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.693945    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:30:21.789469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.789442    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:21.789631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.789498    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:21.823349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.823324    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:21.899346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.899259    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\") pod \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") "
Oct 30 19:30:21.899521 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.899351    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\" (UniqueName: \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\") pod \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\" (UID: \"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc\") "
Oct 30 19:30:21.899748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.899722    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca" pod "cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" (UID: "cba930e3-3e7c-41f1-bbc0-733d44aa5ccc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:21.900335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.900183    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-maas-api-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:21.902414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:21.902362    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp" pod "cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" (UID: "cba930e3-3e7c-41f1-bbc0-733d44aa5ccc"). InnerVolumeSpecName "kube-api-access-n7kzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:22.001661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.001627    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7kzp\" (UniqueName: \"kubernetes.io/projected/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc-kube-api-access-n7kzp\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:22.429189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.429139    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" event={"ID":"aae483f3-d484-4f73-8ec4-dfde145ea837","Type":"ContainerStarted","Data":"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"}
Oct 30 19:30:22.429352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.429206    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" event={"ID":"aae483f3-d484-4f73-8ec4-dfde145ea837","Type":"ContainerStarted","Data":"d6055af97c768c848cf42b8dacda7895fecd1faf63ba5a981d0284b878365af2"}
Oct 30 19:30:22.432221 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.432198    5033 generic.go:356] "Generic (PLEG): container finished" podID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerID="f86344838a183cce37adcb7bf23b6294759d65190567b0734820220bf2fea621" exitCode=60
Oct 30 19:30:22.432373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.432269    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerDied","Data":"f86344838a183cce37adcb7bf23b6294759d65190567b0734820220bf2fea621"}
Oct 30 19:30:22.432373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.432322    5033 scope.go:122] "RemoveContainer" containerID="a0b5f94caf88283952c4ceca1b8356be435bcaae36f5478c6ac5806ad196829e"
Oct 30 19:30:22.432657 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.432641    5033 scope.go:122] "RemoveContainer" containerID="f86344838a183cce37adcb7bf23b6294759d65190567b0734820220bf2fea621"
Oct 30 19:30:22.432955 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:22.432933    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-qshk8_odh-ai-gateway-infra(4961f1da-df1d-4b55-8633-50f7ba990f52)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52"
Oct 30 19:30:22.433797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.433781    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"
Oct 30 19:30:22.433907 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.433825    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d" event={"ID":"cba930e3-3e7c-41f1-bbc0-733d44aa5ccc","Type":"ContainerDied","Data":"d4c41a686b0b56716a88f88b0e39531deb516e40404689c5df809a35582f16ab"}
Oct 30 19:30:22.445470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.445444    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:22.446168 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.446149    5033 scope.go:122] "RemoveContainer" containerID="20cbb5c41e05eef3711401f2fe9ad0080a206a80ad4543448e36437ddefe4b39"
Oct 30 19:30:22.471991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.471934    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podStartSLOduration=2.471917512 podStartE2EDuration="2.471917512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:22.455169316 +0000 UTC m=+687.424810152" watchObservedRunningTime="2026-10-30 19:30:22.471917512 +0000 UTC m=+687.441558347"
Oct 30 19:30:22.515211 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.515172    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:22.518725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.518700    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-6cf459-29889810-ch56d"]
Oct 30 19:30:22.733732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.733709    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:22.799080 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.794423    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:22.799080 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.794479    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:22.834089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.834052    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\") pod \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") "
Oct 30 19:30:22.834259 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.834097    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\") pod \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") "
Oct 30 19:30:22.834322 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.834292    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\" (UniqueName: \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\") pod \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\" (UID: \"5b66f84b-3a5e-450c-9e9b-ee2e3e290628\") "
Oct 30 19:30:22.834499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.834472    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca" pod "5b66f84b-3a5e-450c-9e9b-ee2e3e290628" (UID: "5b66f84b-3a5e-450c-9e9b-ee2e3e290628"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:22.834812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.834789    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:22.841485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.841455    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca" pod "5b66f84b-3a5e-450c-9e9b-ee2e3e290628" (UID: "5b66f84b-3a5e-450c-9e9b-ee2e3e290628"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:22.842869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.842843    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn" pod "5b66f84b-3a5e-450c-9e9b-ee2e3e290628" (UID: "5b66f84b-3a5e-450c-9e9b-ee2e3e290628"). InnerVolumeSpecName "kube-api-access-9smwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:22.936358 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.936324    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:22.936358 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:22.936354    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9smwn\" (UniqueName: \"kubernetes.io/projected/5b66f84b-3a5e-450c-9e9b-ee2e3e290628-kube-api-access-9smwn\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:23.193759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.191071    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" path="/var/lib/kubelet/pods/3576df6c-35f3-4a36-93be-9756930eb7ca/volumes"
Oct 30 19:30:23.193759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.191710    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" path="/var/lib/kubelet/pods/cba930e3-3e7c-41f1-bbc0-733d44aa5ccc/volumes"
Oct 30 19:30:23.235074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.235044    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:23.442148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.442103    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"
Oct 30 19:30:23.442148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.442133    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw" event={"ID":"5b66f84b-3a5e-450c-9e9b-ee2e3e290628","Type":"ContainerDied","Data":"c3f4b4fe3befd08f58af7e882141d5c7e40b30da3cf143bb9f5c6775fb350c7f"}
Oct 30 19:30:23.442338 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.442190    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3f4b4fe3befd08f58af7e882141d5c7e40b30da3cf143bb9f5c6775fb350c7f"
Oct 30 19:30:23.688572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.688546    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:23.745817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.745156    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\" (UniqueName: \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\") pod \"4961f1da-df1d-4b55-8633-50f7ba990f52\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") "
Oct 30 19:30:23.745817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.745226    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\") pod \"4961f1da-df1d-4b55-8633-50f7ba990f52\" (UID: \"4961f1da-df1d-4b55-8633-50f7ba990f52\") "
Oct 30 19:30:23.747116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.746258    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca" pod "4961f1da-df1d-4b55-8633-50f7ba990f52" (UID: "4961f1da-df1d-4b55-8633-50f7ba990f52"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:23.747116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.746586    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4961f1da-df1d-4b55-8633-50f7ba990f52-maas-api-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:23.759560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.749356    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4" pod "4961f1da-df1d-4b55-8633-50f7ba990f52" (UID: "4961f1da-df1d-4b55-8633-50f7ba990f52"). InnerVolumeSpecName "kube-api-access-trkn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:23.789939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.789908    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:30:23.790081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.789968    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 19:30:23.847807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:23.847773    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trkn4\" (UniqueName: \"kubernetes.io/projected/4961f1da-df1d-4b55-8633-50f7ba990f52-kube-api-access-trkn4\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:24.458099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.458073    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"
Oct 30 19:30:24.458819 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.458598    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8" event={"ID":"4961f1da-df1d-4b55-8633-50f7ba990f52","Type":"ContainerDied","Data":"dc1a0dca63cf3d14a68f7f5b9d562ad36937987bd64336391f6a4a818eae2dcc"}
Oct 30 19:30:24.458920 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.458861    5033 scope.go:122] "RemoveContainer" containerID="f86344838a183cce37adcb7bf23b6294759d65190567b0734820220bf2fea621"
Oct 30 19:30:24.496634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.496604    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:24.508451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.502141    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-qshk8"]
Oct 30 19:30:24.829036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.827986    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:24.838699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.829393    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:24.838699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:24.837223    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:30:25.171296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:25.171216    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" path="/var/lib/kubelet/pods/4961f1da-df1d-4b55-8633-50f7ba990f52/volumes"
Oct 30 19:30:25.646695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:25.645243    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:25.661768 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:25.661741    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:26.495317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:26.495291    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.977152    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.977932    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5b66f84b-3a5e-450c-9e9b-ee2e3e290628" containerName="revoke-keys"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.977950    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b66f84b-3a5e-450c-9e9b-ee2e3e290628" containerName="revoke-keys"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.977988    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.977995    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978021    5033 cpu_manager.go:414] "containerMap: removing container" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978028    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978047    5033 cpu_manager.go:414] "containerMap: removing container" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978054    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978070    5033 cpu_manager.go:414] "containerMap: removing container" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978077    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978087    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978094    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978224    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978236    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b66f84b-3a5e-450c-9e9b-ee2e3e290628" containerName="revoke-keys"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978250    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978259    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bc93ffe-e598-4fd4-a5bc-f76819afc7a0" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978270    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978280    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978291    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3576df6c-35f3-4a36-93be-9756930eb7ca" containerName="cleanup"
Oct 30 19:30:27.980015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.978301    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="cba930e3-3e7c-41f1-bbc0-733d44aa5ccc" containerName="cleanup"
Oct 30 19:30:27.987568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:27.987547    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.010640 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.010612    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:30:28.039460 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.039433    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:30:28.040041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.040009    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:28.040041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.040030    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:28.040259 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.040236    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4961f1da-df1d-4b55-8633-50f7ba990f52" containerName="cleanup"
Oct 30 19:30:28.043731 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.043712    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.077318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.077281    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:30:28.112473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112439    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbv8w\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.112626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112480    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.112735 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112643    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.112805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112737    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.112861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112798    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.112923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112871    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.112923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112902    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.113007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.112936    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113028    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.113150 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113132    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113194 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113174    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.113248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113199    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113298    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113352    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.113411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113391    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.113449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113415    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113442    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rj26\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.113549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.113515    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215131 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215104    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215142    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215164    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215362    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215446 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215410    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215446    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215485    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215519    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215530    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215591    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rj26\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215631    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.215870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215655    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215713    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbv8w\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215746    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.215870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215857    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215869    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215930    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.216073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.215979    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216060    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.216202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216091    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.216202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216125    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216141    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216445    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216551 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216513    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.216812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.216788    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.217104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.217082    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.217806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.217354    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.217806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.217477    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.217994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.217976    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.218044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.217982    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.218354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.218337    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.218662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.218645    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.229546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.229525    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.229787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.229768    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rj26\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.229864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.229791    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbv8w\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.235176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.235129    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.308318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.308292    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:28.390381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.385140    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:28.619841 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:28.619807    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834f5518_a544_487b_8c33_1148b603564c.slice/crio-1a38bb007b50168a3e5c96adf4bedf78056f633f98d24fac9bc25d085d14db35 WatchSource:0}: Error finding container 1a38bb007b50168a3e5c96adf4bedf78056f633f98d24fac9bc25d085d14db35: Status 404 returned error can't find the container with id 1a38bb007b50168a3e5c96adf4bedf78056f633f98d24fac9bc25d085d14db35
Oct 30 19:30:28.620039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.620019    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:30:28.622166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.622132    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:28.622247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.622197    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:28.622247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.622229    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:28.703826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.703797    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:30:28.705155 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:28.705129    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34d5d61_7e02_4fcf_9fc9_178f8bc2f032.slice/crio-b50757eccc29191718fcb6210d671ee29e94bf31c704622c559fb3ad4232c9a9 WatchSource:0}: Error finding container b50757eccc29191718fcb6210d671ee29e94bf31c704622c559fb3ad4232c9a9: Status 404 returned error can't find the container with id b50757eccc29191718fcb6210d671ee29e94bf31c704622c559fb3ad4232c9a9
Oct 30 19:30:28.708067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.708027    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:28.708141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.708110    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:28.708192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:28.708177    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:29.529621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:29.529578    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" event={"ID":"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032","Type":"ContainerStarted","Data":"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"}
Oct 30 19:30:29.530136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:29.529640    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" event={"ID":"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032","Type":"ContainerStarted","Data":"b50757eccc29191718fcb6210d671ee29e94bf31c704622c559fb3ad4232c9a9"}
Oct 30 19:30:29.533524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:29.533494    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" event={"ID":"834f5518-a544-487b-8c33-1148b603564c","Type":"ContainerStarted","Data":"5e6959dede6f7f644bc71661398d2243aa865a9e115866bb2eed1fc0e1f16338"}
Oct 30 19:30:29.533659 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:29.533544    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" event={"ID":"834f5518-a544-487b-8c33-1148b603564c","Type":"ContainerStarted","Data":"1a38bb007b50168a3e5c96adf4bedf78056f633f98d24fac9bc25d085d14db35"}
Oct 30 19:30:29.571699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:29.566383    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podStartSLOduration=2.5663622520000002 podStartE2EDuration="2.566362252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:29.557298051 +0000 UTC m=+694.526938887" watchObservedRunningTime="2026-10-30 19:30:29.566362252 +0000 UTC m=+694.536003088"
Oct 30 19:30:30.314887 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.314855    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:30.315293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.315270    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:30.315444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.315423    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:30.396933 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.396887    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:30.399395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.399369    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:30.399612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:30.399587    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:31.237006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.236973    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:31.279083 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.279057    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:31.332700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.324015    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:31.332700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.324071    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:31.332966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.332866    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podStartSLOduration=4.33284656 podStartE2EDuration="4.33284656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:29.600015704 +0000 UTC m=+694.569656541" watchObservedRunningTime="2026-10-30 19:30:31.33284656 +0000 UTC m=+696.302487396"
Oct 30 19:30:31.396823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.396793    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:31.397069 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.397045    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:31.643351 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:31.643274    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:30:32.319701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:32.319467    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:32.319701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:32.319533    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:32.393699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:32.393275    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:32.393699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:32.393355    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:33.309099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.309060    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:33.309399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.309368    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:33.395699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.392041    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:33.395699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.392095    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:33.701756 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.694635    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:30:33.708803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.708776    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.771926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.737099    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-6cf459\"/\"distinct-2-w1-6cf459-kserve-self-signed-certs\""
Oct 30 19:30:33.810710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.793725    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.811007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.810978    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.859820 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.859772    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.884701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.860096    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.885740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.885460    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9csmz\" (UniqueName: \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.885740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.885551    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.898699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.887726    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986451    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986538    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986577    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986664    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986744    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9csmz\" (UniqueName: \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:33.995752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:33.986810    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.080700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.055114    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.080955 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.056050    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.140544 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.140464    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9csmz\" (UniqueName: \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.168698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.148891    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.220699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.215329    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.220699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.218926    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\") pod \"distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") " pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:34.315177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.315083    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:34.315433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.315407    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:34.394697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.391491    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:34.394697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.391553    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:34.469017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:34.468988    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:35.450087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.448813    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:35.450087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.448880    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:35.450087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.449986    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:35.450087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.450031    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:35.540404 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:30:35.540369    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:30:35.584564 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.584202    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:30:35.643784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.638080    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:35.650738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.648735    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:35.673829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.673772    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:35.726267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.726205    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" event={"ID":"f37fc0e1-0948-462d-83c2-64803cd0151f","Type":"ContainerStarted","Data":"1629dec1bd02a3d2cf64ff7d93f1d2a1a1f689aec87e54680d6f328f103f8a47"}
Oct 30 19:30:35.739885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:35.736778    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:30:36.154750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.152420    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:36.311096 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.309619    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:30:36.311096 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.309779    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:30:36.390700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.390236    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:30:36.390700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.390308    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 19:30:36.674024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.673979    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:30:36.678056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.678029    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:30:36.750696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.739845    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:36.879697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.879464    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:30:36.899774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:36.899741    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:37.158332 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.158282    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:30:37.159248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.159212    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:37.614114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.613833    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:37.615305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.615282    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:37.617073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.617049    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:30:37.810397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.810364    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" event={"ID":"f37fc0e1-0948-462d-83c2-64803cd0151f","Type":"ContainerStarted","Data":"8e571525f9002409f6b3780758b3d7c92d053b0aebe49c5d42dffaa86ea7420d"}
Oct 30 19:30:37.838509 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:37.838446    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podStartSLOduration=4.838427166 podStartE2EDuration="4.838427166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:37.834757323 +0000 UTC m=+702.804398159" watchObservedRunningTime="2026-10-30 19:30:37.838427166 +0000 UTC m=+702.808068005"
Oct 30 19:30:38.399698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:38.398748    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:30:38.399698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:38.398811    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:38.471699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:38.470751    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:39.308005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:39.307976    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:39.308518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:39.308423    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:39.309496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:39.309472    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:30:43.220989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.220955    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:30:43.226071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.226047    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.237207 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.237181    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:30:43.318303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318266    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318488 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318422    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7sb4\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318545 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318498    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318594    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318705    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318772    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318817    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.318939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318925    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.319001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.318985    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420168    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420233    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420309    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420348    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420570 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420390    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420570 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420469    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7sb4\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420570 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420498    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420832 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420591    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420832 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420628    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420832 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420759    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420832    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.420991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.420978    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.421099 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.421027    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.421484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.421460    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.422770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.422746    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.423264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.423246    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.433990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.433947    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.434303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.434284    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7sb4\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.545717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.545640    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:43.556295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.556269    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:30:43.561518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.561499    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.571166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.571124    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-9f2d57-openshift-default-dockercfg-l4n9q\""
Oct 30 19:30:43.582289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.581376    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623270    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623360    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623463    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623582    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623644    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623696    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdr5v\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623781    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.623853    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.625075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.624026    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.727658 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.727613    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728303    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728384    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728430    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728464    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728490    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728505    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdr5v\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728539    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.728605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.728585    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.729503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.729444    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.729608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.729538    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.729742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.729718    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.729815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.729722    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.730469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.730208    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.736721 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.733533    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.736721 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.733625    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.738911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.738891    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.756414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.756389    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdr5v\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:43.877892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.877856    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:30:43.883697 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:43.882317    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534601b5_8ae0_46c4_9acb_7e7c9f967cc4.slice/crio-c18285cacdea3efb1168e0bd51df162909468c5a4b581365040216366b2b31ba WatchSource:0}: Error finding container c18285cacdea3efb1168e0bd51df162909468c5a4b581365040216366b2b31ba: Status 404 returned error can't find the container with id c18285cacdea3efb1168e0bd51df162909468c5a4b581365040216366b2b31ba
Oct 30 19:30:43.886867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.885492    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:43.886867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.885570    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:43.886867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.885610    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:43.907311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.907285    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" event={"ID":"534601b5-8ae0-46c4-9acb-7e7c9f967cc4","Type":"ContainerStarted","Data":"c18285cacdea3efb1168e0bd51df162909468c5a4b581365040216366b2b31ba"}
Oct 30 19:30:43.919621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:43.919591    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:44.231263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.231233    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:30:44.232863 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:44.232836    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0b452a_f907_42ec_abe5_8fad195a14c3.slice/crio-4e5eeed3faf5ac92cffbdbd563ccc3cdcc37eae89fb912714d14a54d13a97533 WatchSource:0}: Error finding container 4e5eeed3faf5ac92cffbdbd563ccc3cdcc37eae89fb912714d14a54d13a97533: Status 404 returned error can't find the container with id 4e5eeed3faf5ac92cffbdbd563ccc3cdcc37eae89fb912714d14a54d13a97533
Oct 30 19:30:44.235836 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.235803    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:44.235912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.235869    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:44.235912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.235899    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:44.478701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.476075    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:44.505079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.505007    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:44.926488 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.926396    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" event={"ID":"ca0b452a-f907-42ec-abe5-8fad195a14c3","Type":"ContainerStarted","Data":"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"}
Oct 30 19:30:44.926488 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.926471    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" event={"ID":"ca0b452a-f907-42ec-abe5-8fad195a14c3","Type":"ContainerStarted","Data":"4e5eeed3faf5ac92cffbdbd563ccc3cdcc37eae89fb912714d14a54d13a97533"}
Oct 30 19:30:44.928393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.928368    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" event={"ID":"534601b5-8ae0-46c4-9acb-7e7c9f967cc4","Type":"ContainerStarted","Data":"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"}
Oct 30 19:30:44.940829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:44.940806    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:30:45.035328 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.035272    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podStartSLOduration=2.035258132 podStartE2EDuration="2.035258132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:45.00503102 +0000 UTC m=+709.974671855" watchObservedRunningTime="2026-10-30 19:30:45.035258132 +0000 UTC m=+710.004898963"
Oct 30 19:30:45.545896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.545868    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:45.547238 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.547214    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:45.547368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.547257    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:45.920843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.920763    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:45.922826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.922731    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:45.922826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:45.922780    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:46.546047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:46.546007    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:46.546512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:46.546063    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:46.925292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:46.925200    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:46.925528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:46.925503    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:47.546603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:47.546563    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:47.547029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:47.546632    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:47.921052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:47.920967    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:47.921052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:47.921020    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:48.546963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:48.546927    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:48.547406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:48.546980    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:48.920749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:48.919983    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:48.920749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:48.920031    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:49.546687 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.546653    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:49.546862 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.546720    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:49.635590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.635524    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podStartSLOduration=6.635505257 podStartE2EDuration="6.635505257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:45.109258794 +0000 UTC m=+710.078899630" watchObservedRunningTime="2026-10-30 19:30:49.635505257 +0000 UTC m=+714.605146093"
Oct 30 19:30:49.636924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.636896    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:30:49.643060 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.643040    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.672398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.672369    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9f2d57\"/\"distinct-2-w3-9f2d57-kserve-self-signed-certs\""
Oct 30 19:30:49.763697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.759333    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:30:49.796755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.791482    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.796755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.791551    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.796755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.791618    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.796755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.791746    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.818893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.809980    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlvwz\" (UniqueName: \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.818893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.810039    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.960591 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.960544    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.983696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.973859    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlvwz\" (UniqueName: \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:49.984036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.984011    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.020930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.020893    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.021163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.021144    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.021365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.021348    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.032584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.032133    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.032584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.962601    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.032584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:49.984560    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.047082 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.046992    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.072744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.060202    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlvwz\" (UniqueName: \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.072744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.066012    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\") pod \"distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") " pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.079924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.077891    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:50.079924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.077950    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:50.290254 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.290217    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:50.546079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.546042    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:30:50.546264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.546098    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 19:30:50.633095 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:50.633064    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4418b18_b0ad_44cf_b5f8_2f0a2afeb166.slice/crio-a9c1b9c4a7fb9d7379a33f6d07208dc117aad510103d913cf7191627bbd15ff3 WatchSource:0}: Error finding container a9c1b9c4a7fb9d7379a33f6d07208dc117aad510103d913cf7191627bbd15ff3: Status 404 returned error can't find the container with id a9c1b9c4a7fb9d7379a33f6d07208dc117aad510103d913cf7191627bbd15ff3
Oct 30 19:30:50.636712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.635167    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:30:50.920399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.920308    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:30:50.920399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:50.920378    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 19:30:51.122657 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.122623    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" event={"ID":"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166","Type":"ContainerStarted","Data":"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"}
Oct 30 19:30:51.122854 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.122671    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" event={"ID":"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166","Type":"ContainerStarted","Data":"a9c1b9c4a7fb9d7379a33f6d07208dc117aad510103d913cf7191627bbd15ff3"}
Oct 30 19:30:51.147548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.147494    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podStartSLOduration=2.147476632 podStartE2EDuration="2.147476632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:51.142666979 +0000 UTC m=+716.112307815" watchObservedRunningTime="2026-10-30 19:30:51.147476632 +0000 UTC m=+716.117117468"
Oct 30 19:30:51.299700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.290787    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:30:51.566764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.566183    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:51.570704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.567648    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:51.583699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.583392    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:30:51.924167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.924090    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:51.924608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.924566    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:51.925247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:51.925226    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:30:54.803626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.803587    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:30:54.818733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.813547    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.823833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.823808    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-171808\"/\"distinct-2-w0-171808-kserve-self-signed-certs\""
Oct 30 19:30:54.833000 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.826851    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:30:54.910903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.910871    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.911156 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.911129    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.911255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.911235    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.911319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.911295    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.911377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.911317    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:54.911438 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:54.911404    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lc26\" (UniqueName: \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018026    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018120    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018183    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018274    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lc26\" (UniqueName: \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018334    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.018639    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.020479    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.020699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.020696    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.021242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.020991    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.043075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.043042    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.045469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.045428    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.052433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.052406    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lc26\" (UniqueName: \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\") pod \"distinct-2-w0-171808-kserve-5db94578b7-tp2qg\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") " pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.159053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.158985    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:55.480475 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:55.480442    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:30:56.189641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:56.189600    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" event={"ID":"e01494b8-4ee0-4097-88ef-7944f98cfc39","Type":"ContainerStarted","Data":"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"}
Oct 30 19:30:56.190268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:56.190238    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" event={"ID":"e01494b8-4ee0-4097-88ef-7944f98cfc39","Type":"ContainerStarted","Data":"b7c75e3a481f67da2abb96b3a0a03157b200ae36cb6984855c3712e9007d1593"}
Oct 30 19:30:56.225261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:56.225193    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podStartSLOduration=2.225170027 podStartE2EDuration="2.225170027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:56.220102998 +0000 UTC m=+721.189743833" watchObservedRunningTime="2026-10-30 19:30:56.225170027 +0000 UTC m=+721.194810862"
Oct 30 19:30:57.160700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.160084    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:30:57.405708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.405069    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:57.494702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.489841    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:30:57.552698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.545759    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:57.576695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.573343    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:57.576695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.573446    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:30:57.647699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.646756    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:30:57.748748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.747752    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:57.937696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:57.936755    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:58.418214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.415397    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:30:58.421500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.421482    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.474217 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.474137    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:30:58.478749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.478721    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:30:58.498871 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.498817    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:30:58.499019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.498936    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.549917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.549881    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs7cg\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550080 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.549969    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550120    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550166    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550199    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550234    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550258    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550288    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.550419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.550339    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652070 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652041    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652070 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652072    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652094    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652318    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652392 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652364    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652447 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652425    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652568    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652602    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652644    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652671    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652732    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rfr7\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652767    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652821    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652870    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652900    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.652947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652925    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.652947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.652937    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.653264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653144    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.653264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653155    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs7cg\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.653264 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653219    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.653432 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653267    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.653669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653644    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.653882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.653862    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.655564 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.655545    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.655980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.655952    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.683667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.683606    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs7cg\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.684341 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.684318    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.755031 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755002    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755048    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755078    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rfr7\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755146    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755191    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755290    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755375    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755405    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755422    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755435    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755541    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755824    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.755930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.755902    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.756517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.756490    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.758104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.758080    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.758932 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.758907    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.762994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.762974    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:58.796582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.796555    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.797167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.797145    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rfr7\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\") pod \"e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") " pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:58.819794 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:58.819759    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:30:59.054030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.054002    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:30:59.054575 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:59.054543    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fde99e3_88c5_44d9_bc8c_20173f17f42b.slice/crio-f216130d2ad1be83e3a072dd7d26b0ee5ca4eba1213630f50dbea0b255509c32 WatchSource:0}: Error finding container f216130d2ad1be83e3a072dd7d26b0ee5ca4eba1213630f50dbea0b255509c32: Status 404 returned error can't find the container with id f216130d2ad1be83e3a072dd7d26b0ee5ca4eba1213630f50dbea0b255509c32
Oct 30 19:30:59.057219 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.057185    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.057318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.057255    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.057318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.057284    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.119919 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:30:59.119707    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27fa030_73b4_4864_8aaf_29bc3451860d.slice/crio-2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0 WatchSource:0}: Error finding container 2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0: Status 404 returned error can't find the container with id 2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0
Oct 30 19:30:59.121927 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.121583    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:30:59.124479 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.123836    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.124479 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.123938    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.124479 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.123998    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:59.389017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.388935    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" event={"ID":"a27fa030-73b4-4864-8aaf-29bc3451860d","Type":"ContainerStarted","Data":"034bab3d26dc39f031e1391260eee6a407119bac1593d7f3bb05b6171331c60c"}
Oct 30 19:30:59.389017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.388992    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" event={"ID":"a27fa030-73b4-4864-8aaf-29bc3451860d","Type":"ContainerStarted","Data":"2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0"}
Oct 30 19:30:59.390748 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.390719    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" event={"ID":"2fde99e3-88c5-44d9-bc8c-20173f17f42b","Type":"ContainerStarted","Data":"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"}
Oct 30 19:30:59.390860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.390760    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" event={"ID":"2fde99e3-88c5-44d9-bc8c-20173f17f42b","Type":"ContainerStarted","Data":"f216130d2ad1be83e3a072dd7d26b0ee5ca4eba1213630f50dbea0b255509c32"}
Oct 30 19:30:59.421172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.421115    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podStartSLOduration=1.421085221 podStartE2EDuration="1.421085221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:59.41774632 +0000 UTC m=+724.387387166" watchObservedRunningTime="2026-10-30 19:30:59.421085221 +0000 UTC m=+724.390726057"
Oct 30 19:30:59.443989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.443922    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podStartSLOduration=1.443904552 podStartE2EDuration="1.443904552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:59.440005894 +0000 UTC m=+724.409646730" watchObservedRunningTime="2026-10-30 19:30:59.443904552 +0000 UTC m=+724.413545388"
Oct 30 19:30:59.765835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.765797    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:30:59.824786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:30:59.824753    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:31:00.297702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.294753    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:00.326253 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.326224    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:00.448195 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.448152    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:00.788815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.788777    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:00.790060 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.790026    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:00.826797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.826746    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:00.827074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:00.827032    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:01.764050 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:01.764014    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:01.764656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:01.764623    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:01.827132 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:01.827095    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:01.827409 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:01.827378    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:02.770707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:02.767005    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:02.770707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:02.767065    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:02.821196 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:02.821162    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:02.821387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:02.821216    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:03.771051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:03.771013    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:03.771647 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:03.771612    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:03.821696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:03.821196    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:03.821696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:03.821270    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:04.765701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:04.763993    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:04.765701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:04.764059    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:04.820692 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:04.820652    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:04.821064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:04.820739    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:05.049270 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:31:05.044555    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:31:05.203142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.203100    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:31:05.208835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.208805    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:31:05.484375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.484349    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:31:05.763637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.763551    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:31:05.763843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.763631    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 19:31:05.821311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.821275    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:31:05.821908 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:05.821877    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 19:31:06.774050 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.774020    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:31:06.775104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.775078    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:31:06.776032 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.776010    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:31:06.825568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.825535    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:31:06.826079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.826015    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:31:06.826812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:06.826787    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:31:08.663491 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.663457    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"]
Oct 30 19:31:08.670111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.670083    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.676117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.675269    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"]
Oct 30 19:31:08.677243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.677105    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:31:08.677243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.677217    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:31:08.705429 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.705402    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2lpp\" (UniqueName: \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.705580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.705434    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.705580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.705454    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.806687 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.806643    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2lpp\" (UniqueName: \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.806821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.806691    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.806821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.806713    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.807444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.807425    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.809254 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.809228    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.816310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.816288    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2lpp\" (UniqueName: \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\") pod \"maas-api-revoke-sub-4a5a6fd7-6hx7n\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:08.991193 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:08.991160    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:09.300485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:09.300451    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"]
Oct 30 19:31:09.303563 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:09.303516    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod549c5159_aa5b_42db_b11a_d725abeff58b.slice/crio-46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5 WatchSource:0}: Error finding container 46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5: Status 404 returned error can't find the container with id 46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5
Oct 30 19:31:09.515056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:09.515022    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n" event={"ID":"549c5159-aa5b-42db-b11a-d725abeff58b","Type":"ContainerStarted","Data":"46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5"}
Oct 30 19:31:10.523039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:10.522995    5033 generic.go:356] "Generic (PLEG): container finished" podID="549c5159-aa5b-42db-b11a-d725abeff58b" containerID="c84706125dc67da0043b8483dd85b82a8d61bcd1bb9fd636d1f6b10f2f865514" exitCode=0
Oct 30 19:31:10.523413 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:10.523080    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n" event={"ID":"549c5159-aa5b-42db-b11a-d725abeff58b","Type":"ContainerDied","Data":"c84706125dc67da0043b8483dd85b82a8d61bcd1bb9fd636d1f6b10f2f865514"}
Oct 30 19:31:11.723345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.723320    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:11.750387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.750358    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\") pod \"549c5159-aa5b-42db-b11a-d725abeff58b\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") "
Oct 30 19:31:11.750600 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.750584    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\") pod \"549c5159-aa5b-42db-b11a-d725abeff58b\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") "
Oct 30 19:31:11.750734 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.750612    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\" (UniqueName: \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\") pod \"549c5159-aa5b-42db-b11a-d725abeff58b\" (UID: \"549c5159-aa5b-42db-b11a-d725abeff58b\") "
Oct 30 19:31:11.751020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.750993    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca" pod "549c5159-aa5b-42db-b11a-d725abeff58b" (UID: "549c5159-aa5b-42db-b11a-d725abeff58b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:11.751283 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.751261    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:11.752876 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.752845    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca" pod "549c5159-aa5b-42db-b11a-d725abeff58b" (UID: "549c5159-aa5b-42db-b11a-d725abeff58b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:11.753864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.753840    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp" pod "549c5159-aa5b-42db-b11a-d725abeff58b" (UID: "549c5159-aa5b-42db-b11a-d725abeff58b"). InnerVolumeSpecName "kube-api-access-c2lpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:11.853183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.852591    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/549c5159-aa5b-42db-b11a-d725abeff58b-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:11.853183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:11.852634    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2lpp\" (UniqueName: \"kubernetes.io/projected/549c5159-aa5b-42db-b11a-d725abeff58b-kube-api-access-c2lpp\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:12.541918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:12.541888    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"
Oct 30 19:31:12.542082 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:12.541891    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n" event={"ID":"549c5159-aa5b-42db-b11a-d725abeff58b","Type":"ContainerDied","Data":"46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5"}
Oct 30 19:31:12.542082 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:12.542016    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46a9007c1d9fc480c04a8e35112a75dbb9762ee98c170797de1b8f25d3508cb5"
Oct 30 19:31:13.059396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.059359    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:31:13.060045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.060029    5033 cpu_manager.go:414] "containerMap: removing container" podUID="549c5159-aa5b-42db-b11a-d725abeff58b" containerName="revoke-keys"
Oct 30 19:31:13.060108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.060049    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="549c5159-aa5b-42db-b11a-d725abeff58b" containerName="revoke-keys"
Oct 30 19:31:13.060184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.060173    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="549c5159-aa5b-42db-b11a-d725abeff58b" containerName="revoke-keys"
Oct 30 19:31:13.064142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.064122    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.080767 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.080746    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:31:13.170639 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.170605    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.170838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.170649    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.170901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.170847    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hhxn\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171059 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171031    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171157    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171188    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171250    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171318    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.171466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.171421    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273211 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273177    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273236    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273284    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273314    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273379    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273412    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273449    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273491    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hhxn\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273625    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.273864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273840    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.274055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.273873    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.278535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.274932    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.278535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.275168    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.278535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.275258    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.278535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.276373    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.278535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.276888    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.287135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.287104    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hhxn\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.287292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.287274    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\") pod \"e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") " pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.384332 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.383842    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:13.614213 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.609166    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:31:13.625630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.625547    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.652053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.651949    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714111    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714168    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714223    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714292    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714345    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714444    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714489    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714551    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvxmw\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.714867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.714649    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.822422 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.822382    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.822656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.822632    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.822881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.822857    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvxmw\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.823091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.823059    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.823262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.823243    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.823400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.823384    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.823578 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.823529    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.824906 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.824885    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.825087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.825068    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.825724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.825670    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.826534 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.826509    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.827311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.827287    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.828414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.828107    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.828583 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.828341    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.831552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.831530    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.845232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.845177    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.859434 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.859373    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:13.897744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:13.888292    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvxmw\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\") pod \"e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") " pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:14.088281 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.087994    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:14.245699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.213142    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:31:14.268763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.237707    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:14.269115 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.269076    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:14.269299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.269277    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:14.753024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:14.752988    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" event={"ID":"cdde0ee1-e242-49f3-a81c-6241adfa007c","Type":"ContainerStarted","Data":"81052f046e5f189fff25342a080d412ec06e5fb896d4bd25cb0f445cd09278ba"}
Oct 30 19:31:15.485768 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:15.417961    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:31:15.523700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:15.502152    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:15.750019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:15.749915    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.013764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.012795    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.048400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.048336    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:16.063026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.052238    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:16.096805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.050749    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:31:16.138612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.138560    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:16.190418 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.190275    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.217704 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:31:16.192964    5033 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.004s"
Oct 30 19:31:16.217704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.193787    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/health\": context deadline exceeded"
Oct 30 19:31:16.238971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.237362    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:31:16.416746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.415747    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.479706 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.476767    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.479706 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.477916    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:31:16.512705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.509770    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.512705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.510248    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:31:16.526585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.525940    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.542707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.537042    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" event={"ID":"cdde0ee1-e242-49f3-a81c-6241adfa007c","Type":"ContainerStarted","Data":"f6ca135896633d11d6fe96ad81313aa4eace07353e14ad42a60cffbf568672e5"}
Oct 30 19:31:16.543308 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.543281    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:16.543520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.543498    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.553771 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.553745    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" event={"ID":"9198743c-05d3-413a-a0b5-31ce2bf40c8a","Type":"ContainerStarted","Data":"c01ea3123d0d0fa2dffebc19adf5bcc0020a60c3c3d49e9a4eacb5f44655449a"}
Oct 30 19:31:16.581704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.579238    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.669757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.639041    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.790559 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.650716    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.791203 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.650746    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:31:16.791597 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.662808    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.791989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.731145    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:16.801103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:16.755244    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.198315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.195761    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.198315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.196154    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:17.198315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.196447    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:31:17.198315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.198240    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.259031 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.258387    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:31:17.270831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.270804    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:31:17.271666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.271647    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:31:17.272424 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.272408    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:31:17.549845 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.319284    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podStartSLOduration=4.319264649 podStartE2EDuration="4.319264649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:16.637974404 +0000 UTC m=+741.607615240" watchObservedRunningTime="2026-10-30 19:31:17.319264649 +0000 UTC m=+742.288905488"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.406196    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.406265    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.575503    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.481428    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.418391    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.520061    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.576808    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.582082    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.584719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.582438    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:31:17.708494 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.708456    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" event={"ID":"9198743c-05d3-413a-a0b5-31ce2bf40c8a","Type":"ContainerStarted","Data":"4ac8f0b61e5f92733851ca600bf100b7df96abe3428e260a560f93551b255593"}
Oct 30 19:31:17.813754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.811727    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:31:17.886700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.859965    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:31:17.886700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.860281    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:31:17.886700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:17.860344    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:31:18.106885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.106792    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.116379 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.115943    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:18.164845 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.164804    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:18.165038 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.164884    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:18.216818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.214910    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.243700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.234848    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:18.500699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.498824    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:18.500699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.498883    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:18.509695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.504961    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podStartSLOduration=5.5049416650000005 podStartE2EDuration="5.504941665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:17.888487457 +0000 UTC m=+742.858128293" watchObservedRunningTime="2026-10-30 19:31:18.504941665 +0000 UTC m=+743.474582502"
Oct 30 19:31:18.610029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.608808    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.621623 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.610599    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.621623 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.620103    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:31:18.646791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.646753    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.667393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.667359    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.840754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.811422    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:31:18.871696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.870961    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.963696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:18.962817    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:19.212953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.212925    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:31:19.224354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.224323    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:19.224560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.224539    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:19.302701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.246260    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.381780    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.381885    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.381994    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.382057    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.382111    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:31:19.386703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.382162    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:31:19.411751 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.407293    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:19.411751 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:19.407355    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:20.112698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:20.110040    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:20.112698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:20.110101    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:20.393776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:20.393411    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:20.393776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:20.393470    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:21.114708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.114005    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:21.114708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.114064    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:21.414493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.414412    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:21.414750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.414727    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:21.730141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.730072    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:31:21.734931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.734908    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.742788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.741936    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-639c15\"/\"distinct-2-w6-639c15-kserve-self-signed-certs\""
Oct 30 19:31:21.839718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.839652    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.950441    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.950546    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.950587    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxk9j\" (UniqueName: \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.950653    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.955848    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:21.963723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:21.955964    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.063593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.063498    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.064435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.064411    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.064959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.064941    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxk9j\" (UniqueName: \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.065423 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.065404    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.065998 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.064237    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.066149 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.064869    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.066739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.065911    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.066877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.065971    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.067105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.067089    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.110624 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.110206    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:22.110624 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.110262    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:22.115333 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.113489    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.160336 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.138516    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxk9j\" (UniqueName: \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.230373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.230336    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\") pod \"distinct-2-w6-639c15-kserve-7479c875bf-g8jgq\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") " pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.290701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.289338    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:22.420708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.420131    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:22.420708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.420191    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:22.932571 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:22.932531    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:31:23.114701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.111721    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:23.114701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.111784    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:23.323702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.323139    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" event={"ID":"34d2be18-505a-416f-aec5-b0363d0f75d4","Type":"ContainerStarted","Data":"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"}
Oct 30 19:31:23.323702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.323215    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" event={"ID":"34d2be18-505a-416f-aec5-b0363d0f75d4","Type":"ContainerStarted","Data":"0bf6288d87e45bb582607d474b8693a530e1febf6df0c479e1fdb7c33c507df0"}
Oct 30 19:31:23.364699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.361439    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podStartSLOduration=2.361419851 podStartE2EDuration="2.361419851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:23.359236162 +0000 UTC m=+748.328876999" watchObservedRunningTime="2026-10-30 19:31:23.361419851 +0000 UTC m=+748.331060688"
Oct 30 19:31:23.396977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.395826    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:23.396977 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:23.395883    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:24.110468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:24.110432    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:24.110650 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:24.110487    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:24.298715 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:24.291296    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:24.393094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:24.393010    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:31:24.393665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:24.393636    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 19:31:25.110072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:25.110038    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 19:31:25.110399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:25.110090    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 19:31:25.397989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:25.397911    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:25.398796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:25.398766    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:25.399552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:25.399529    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:31:26.587136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:26.587105    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:26.587802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:26.587775    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:26.588657 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:26.588633    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:31:28.634339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.634300    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:31:28.641828 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.641799    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.644904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.644879    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:31:28.724210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.722750    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:31:28.728556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.728532    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.758445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.758417    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:31:28.805929 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.805901    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpt8f\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.805982    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806150    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806197    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806344 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806326    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806388    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806453    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806473    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.806623 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.806603    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.907791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.907705    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpt8f\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.907791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.907765    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.907877    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908004    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908103    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpwxq\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908134    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908159    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908214    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908292    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908316    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908334    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908369    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908510 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908450    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.908510 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908500    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908622 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908545    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908718    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908851    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.908877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908872    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.909073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908880    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.909073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908928    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.909073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908980    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.909073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.908985    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.909273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.909093    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:28.911365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.911339    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.911637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.911609    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.918009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.917990    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpt8f\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:28.918330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:28.918297    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\") pod \"e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") " pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:29.008370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.008343    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:29.010010 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.009984    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010047    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010094    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010216    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpwxq\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010251    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010367 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010291    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010367 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010338    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010459    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010507    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.010660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010551    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.011013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010990    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.011088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.010999    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.011088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.011054    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.011345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.011321    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.012616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.012561    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.013180 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.013160    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.020257 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.020219    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpwxq\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.020572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.020552    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\") pod \"e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") " pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.063971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.063944    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:29.310308 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.310274    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:31:29.310430 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:29.310274    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca995be6_88c2_474e_ab02_7b6fa35f4c9c.slice/crio-b1183f30ba2eac71a2e90eb8286df7f0eb2f41a782bc0551c5e3d866bbd28897 WatchSource:0}: Error finding container b1183f30ba2eac71a2e90eb8286df7f0eb2f41a782bc0551c5e3d866bbd28897: Status 404 returned error can't find the container with id b1183f30ba2eac71a2e90eb8286df7f0eb2f41a782bc0551c5e3d866bbd28897
Oct 30 19:31:29.313166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.313131    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:29.313262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.313196    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:29.313262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.313226    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:29.375064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.374990    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" event={"ID":"ca995be6-88c2-474e-ab02-7b6fa35f4c9c","Type":"ContainerStarted","Data":"b1183f30ba2eac71a2e90eb8286df7f0eb2f41a782bc0551c5e3d866bbd28897"}
Oct 30 19:31:29.397510 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:29.397485    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901b2c57_dae5_418e_9273_4701dfbcaa26.slice/crio-055a9a22ff0462f8c31e0f00650c0a73e0db3db2f66d993c5ca6ced7b24c80d2 WatchSource:0}: Error finding container 055a9a22ff0462f8c31e0f00650c0a73e0db3db2f66d993c5ca6ced7b24c80d2: Status 404 returned error can't find the container with id 055a9a22ff0462f8c31e0f00650c0a73e0db3db2f66d993c5ca6ced7b24c80d2
Oct 30 19:31:29.397812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.397769    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:31:29.400208 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.400172    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:29.400289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.400261    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:29.400326 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:29.400302    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:30.408345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:30.408307    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" event={"ID":"ca995be6-88c2-474e-ab02-7b6fa35f4c9c","Type":"ContainerStarted","Data":"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"}
Oct 30 19:31:30.424451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:30.424416    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" event={"ID":"901b2c57-dae5-418e-9273-4701dfbcaa26","Type":"ContainerStarted","Data":"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"}
Oct 30 19:31:30.424725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:30.424705    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" event={"ID":"901b2c57-dae5-418e-9273-4701dfbcaa26","Type":"ContainerStarted","Data":"055a9a22ff0462f8c31e0f00650c0a73e0db3db2f66d993c5ca6ced7b24c80d2"}
Oct 30 19:31:30.443359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:30.443301    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podStartSLOduration=2.443286457 podStartE2EDuration="2.443286457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:30.439735401 +0000 UTC m=+755.409376235" watchObservedRunningTime="2026-10-30 19:31:30.443286457 +0000 UTC m=+755.412927291"
Oct 30 19:31:31.015698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.014754    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:31.015698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.014916    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:31.015698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.014955    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:31.069699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.069630    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:31.069890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.069790    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:31.069890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.069837    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:31.782636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.782576    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podStartSLOduration=3.782555372 podStartE2EDuration="3.782555372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:30.470880775 +0000 UTC m=+755.440521613" watchObservedRunningTime="2026-10-30 19:31:31.782555372 +0000 UTC m=+756.752196207"
Oct 30 19:31:31.791696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.783668    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"]
Oct 30 19:31:31.805922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.805897    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:31.818521 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.818493    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:31:31.827963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.827937    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:31:31.835495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.835468    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"]
Oct 30 19:31:31.932610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.932569    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:31.933135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.933113    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn6qf\" (UniqueName: \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:31.933294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:31.933280    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.009122 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.009091    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:32.009342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.009322    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:32.034519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.034439    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.034736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.034719    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn6qf\" (UniqueName: \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.034822 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.034764    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.035109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.035086    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.037104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.037083    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.044170 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.044146    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn6qf\" (UniqueName: \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\") pod \"maas-api-revoke-sub-12c19f23-tbv7m\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.068939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.068534    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:32.068939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.068586    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:32.145663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.145634    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:32.300471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.293278    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:32.349384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.349361    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:32.547936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.546721    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"]
Oct 30 19:31:32.632602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:32.631370    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:33.012848 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.012415    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:33.012848 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.012476    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:33.071740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.067390    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:33.071740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.067447    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:33.542174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.542133    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m" event={"ID":"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd","Type":"ContainerStarted","Data":"5674d33a33ef96e993ba383e0092f25e981f2aaf43244fd39f69619d6014d5e4"}
Oct 30 19:31:33.542334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.542190    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m" event={"ID":"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd","Type":"ContainerStarted","Data":"e3c36b5ca55082c5d11ba47b9bb366ec6fe29c008cc2c65294a3fe2d03e05ff3"}
Oct 30 19:31:33.582174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:33.580338    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m" podStartSLOduration=2.580320809 podStartE2EDuration="2.580320809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:33.56561668 +0000 UTC m=+758.535257517" watchObservedRunningTime="2026-10-30 19:31:33.580320809 +0000 UTC m=+758.549961645"
Oct 30 19:31:34.020696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.014025    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:34.020696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.014080    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:34.068711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.068029    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:34.068711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.068094    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:34.573439 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.573402    5033 generic.go:356] "Generic (PLEG): container finished" podID="59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" containerID="5674d33a33ef96e993ba383e0092f25e981f2aaf43244fd39f69619d6014d5e4" exitCode=0
Oct 30 19:31:34.574178 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:34.574002    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m" event={"ID":"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd","Type":"ContainerDied","Data":"5674d33a33ef96e993ba383e0092f25e981f2aaf43244fd39f69619d6014d5e4"}
Oct 30 19:31:35.017701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:35.013052    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:35.017701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:35.013125    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:35.077703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:35.068183    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:35.077703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:35.068266    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:35.127699 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:31:35.126855    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:31:35.894885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:35.894858    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:36.008799 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.008758    5033 patch_prober.go:29] interesting pod/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:31:36.009044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.008822    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 19:31:36.030993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.030961    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\") pod \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") "
Oct 30 19:31:36.031171 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.031042    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\") pod \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") "
Oct 30 19:31:36.031248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.031204    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\" (UniqueName: \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\") pod \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\" (UID: \"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd\") "
Oct 30 19:31:36.031452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.031424    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca" pod "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" (UID: "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:36.031867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.031846    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:36.033808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.033784    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf" pod "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" (UID: "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd"). InnerVolumeSpecName "kube-api-access-gn6qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:36.033888 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.033782    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca" pod "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" (UID: "59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:36.066241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.066213    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:31:36.066355 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.066269    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 19:31:36.133347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.133265    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:36.133347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.133300    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn6qf\" (UniqueName: \"kubernetes.io/projected/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd-kube-api-access-gn6qf\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:36.437739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.437560    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"]
Oct 30 19:31:36.438656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.438631    5033 cpu_manager.go:414] "containerMap: removing container" podUID="59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" containerName="revoke-keys"
Oct 30 19:31:36.438656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.438657    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" containerName="revoke-keys"
Oct 30 19:31:36.438899 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.438840    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" containerName="revoke-keys"
Oct 30 19:31:36.444116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.444094    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.467370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.448755    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"]
Oct 30 19:31:36.541699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.540880    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.541699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.541033    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hggx9\" (UniqueName: \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.541699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.541248    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.628520 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.628475    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m" event={"ID":"59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd","Type":"ContainerDied","Data":"e3c36b5ca55082c5d11ba47b9bb366ec6fe29c008cc2c65294a3fe2d03e05ff3"}
Oct 30 19:31:36.628953 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.628924    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3c36b5ca55082c5d11ba47b9bb366ec6fe29c008cc2c65294a3fe2d03e05ff3"
Oct 30 19:31:36.629188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.629172    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"
Oct 30 19:31:36.644587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.644552    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.644759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.644618    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hggx9\" (UniqueName: \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.644759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.644728    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.645776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.645748    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.649022 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.649001    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.656103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.656081    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hggx9\" (UniqueName: \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\") pod \"maas-api-revoke-sub-7b1467a7-2drhl\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:36.773863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:36.773829    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:37.196387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.194140    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:37.196387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.194455    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"]
Oct 30 19:31:37.196387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.196084    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:37.203732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.201572    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:31:37.358196 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.355075    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:37.359403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.359380    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:37.360350 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.360329    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:31:37.637733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:37.637702    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl" event={"ID":"c38fcf53-0871-4344-87f9-81bb0bdb3183","Type":"ContainerStarted","Data":"40e21f8a9d622055a66a88fbbd4e4116b3f59bdef25dafccde43b8d94cb0455e"}
Oct 30 19:31:38.648027 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:38.647951    5033 generic.go:356] "Generic (PLEG): container finished" podID="c38fcf53-0871-4344-87f9-81bb0bdb3183" containerID="bb31177bd836de6e8feca5872ef778cd15ce31bf3d87af1b286c21c0ab7dc24e" exitCode=0
Oct 30 19:31:38.648421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:38.648042    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl" event={"ID":"c38fcf53-0871-4344-87f9-81bb0bdb3183","Type":"ContainerDied","Data":"bb31177bd836de6e8feca5872ef778cd15ce31bf3d87af1b286c21c0ab7dc24e"}
Oct 30 19:31:39.900463 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:39.900431    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:40.044421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.044381    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\") pod \"c38fcf53-0871-4344-87f9-81bb0bdb3183\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") "
Oct 30 19:31:40.044650 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.044633    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\" (UniqueName: \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\") pod \"c38fcf53-0871-4344-87f9-81bb0bdb3183\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") "
Oct 30 19:31:40.044797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.044667    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\") pod \"c38fcf53-0871-4344-87f9-81bb0bdb3183\" (UID: \"c38fcf53-0871-4344-87f9-81bb0bdb3183\") "
Oct 30 19:31:40.045137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.045106    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca" pod "c38fcf53-0871-4344-87f9-81bb0bdb3183" (UID: "c38fcf53-0871-4344-87f9-81bb0bdb3183"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:40.045309 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.045286    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:40.046598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.046576    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca" pod "c38fcf53-0871-4344-87f9-81bb0bdb3183" (UID: "c38fcf53-0871-4344-87f9-81bb0bdb3183"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:40.047152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.047135    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9" pod "c38fcf53-0871-4344-87f9-81bb0bdb3183" (UID: "c38fcf53-0871-4344-87f9-81bb0bdb3183"). InnerVolumeSpecName "kube-api-access-hggx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:40.147214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.147187    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hggx9\" (UniqueName: \"kubernetes.io/projected/c38fcf53-0871-4344-87f9-81bb0bdb3183-kube-api-access-hggx9\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:40.147214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.147213    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c38fcf53-0871-4344-87f9-81bb0bdb3183-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:40.669006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.668942    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl" event={"ID":"c38fcf53-0871-4344-87f9-81bb0bdb3183","Type":"ContainerDied","Data":"40e21f8a9d622055a66a88fbbd4e4116b3f59bdef25dafccde43b8d94cb0455e"}
Oct 30 19:31:40.669293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.669016    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40e21f8a9d622055a66a88fbbd4e4116b3f59bdef25dafccde43b8d94cb0455e"
Oct 30 19:31:40.669293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:40.669082    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"
Oct 30 19:31:43.187753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.187718    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"]
Oct 30 19:31:43.188444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.188426    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c38fcf53-0871-4344-87f9-81bb0bdb3183" containerName="revoke-keys"
Oct 30 19:31:43.188444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.188444    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c38fcf53-0871-4344-87f9-81bb0bdb3183" containerName="revoke-keys"
Oct 30 19:31:43.188584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.188575    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="c38fcf53-0871-4344-87f9-81bb0bdb3183" containerName="revoke-keys"
Oct 30 19:31:43.192395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.192377    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.198606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.198510    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:31:43.200138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.200101    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:31:43.202177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.202153    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"]
Oct 30 19:31:43.211007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.210986    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4hn9\" (UniqueName: \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.211233 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.211210    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.211474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.211456    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.313691 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.313631    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hn9\" (UniqueName: \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.313858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.313772    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.313952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.313881    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.314598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.314571    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.316285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.316263    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.323481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.322790    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:31:43.325825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.325803    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hn9\" (UniqueName: \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\") pod \"maas-api-revoke-sub-ec0d47cf-mxgb2\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.328231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.328211    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.343489 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.343447    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:31:43.416179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416143    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416322 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416201    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416453    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416501    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvmc\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416605    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416762    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416796    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.416951 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.416934    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.417038 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.417017    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.512315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.512292    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:43.518367 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518338    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.518499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518387    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvmc\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.518499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518421    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.518580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518515    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.518580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518539    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.518759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.518660    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.519155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.519135    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.519546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.519526    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.519947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.519924    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.527999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.522220    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.527999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.522286    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.527999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.522334    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.527999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.522379    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.527999 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.527173    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.538182 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.536151    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.538938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.538544    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvmc\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.538938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.538885    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.565081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.564074    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.664980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.664763    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:43.858109 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.858070    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"]
Oct 30 19:31:43.989666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:43.989636    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:31:44.059359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.059320    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:44.059464 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.059403    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:44.059464 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.059436    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:44.760698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.760055    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" event={"ID":"97fbfe14-3068-48aa-961e-3f38960ce9ab","Type":"ContainerStarted","Data":"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"}
Oct 30 19:31:44.760698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.760174    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" event={"ID":"97fbfe14-3068-48aa-961e-3f38960ce9ab","Type":"ContainerStarted","Data":"72e0c1e58e1e9f9130d564d304e88136cc8464d34f4c25b0426e3241ba489666"}
Oct 30 19:31:44.814706 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.812726    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2" event={"ID":"4194440b-9220-4f02-b801-caabfe3b3034","Type":"ContainerStarted","Data":"ed5d92682f5744f5de313c1d09a005640d78a5a051e91355b79ec7570691e309"}
Oct 30 19:31:44.814706 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.812797    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2" event={"ID":"4194440b-9220-4f02-b801-caabfe3b3034","Type":"ContainerStarted","Data":"1d848dcd7ac2722fb04be6f0434cb9b4a17fff19fcffb79754d2e4714a7694b9"}
Oct 30 19:31:44.838692 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.828078    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podStartSLOduration=1.828059079 podStartE2EDuration="1.828059079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:44.810582666 +0000 UTC m=+769.780223502" watchObservedRunningTime="2026-10-30 19:31:44.828059079 +0000 UTC m=+769.797699917"
Oct 30 19:31:44.921690 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:44.918814    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2" podStartSLOduration=1.918791138 podStartE2EDuration="1.918791138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:44.896950429 +0000 UTC m=+769.866591265" watchObservedRunningTime="2026-10-30 19:31:44.918791138 +0000 UTC m=+769.888431973"
Oct 30 19:31:45.650702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.650614    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:31:45.672701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.668130    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:45.672701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.669022    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:45.672701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.669163    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:45.672701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.669202    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:45.673079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.672733    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:31:45.877699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.876768    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:45.881746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.880879    5033 generic.go:356] "Generic (PLEG): container finished" podID="4194440b-9220-4f02-b801-caabfe3b3034" containerID="ed5d92682f5744f5de313c1d09a005640d78a5a051e91355b79ec7570691e309" exitCode=0
Oct 30 19:31:45.882137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.882065    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2" event={"ID":"4194440b-9220-4f02-b801-caabfe3b3034","Type":"ContainerDied","Data":"ed5d92682f5744f5de313c1d09a005640d78a5a051e91355b79ec7570691e309"}
Oct 30 19:31:45.930753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:45.929755    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.186205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.185774    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.234705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.225934    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.235739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.234971    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.331815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.331772    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:46.491876 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.491821    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:31:46.552700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.546637    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.591797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.590765    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:31:46.592140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.592051    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.621222 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.618763    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.629267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.629230    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.657174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.654756    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.657174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.655110    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/health\": context deadline exceeded"
Oct 30 19:31:46.657174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.655502    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:31:46.674225 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.674189    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.675498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.675064    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:46.676543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.676517    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:46.676708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.676096    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:31:46.769550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.767011    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:46.769550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.769435    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 19:31:46.974884 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:46.973968    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 30 19:31:47.012847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.011131    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.037746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.033785    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.037746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.034634    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:31:47.037746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.034726    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:31:47.340750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.333786    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:47.369385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.359552    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.369385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.361069    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.495117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.492624    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:47.558702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.558651    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:31:47.597947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.591788    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:31:47.597947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.591843    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:31:47.597947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.567278    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:47.635823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.635787    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:47.676618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.653167    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.676618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.653441    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.676618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.653663    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:31:47.676618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.653746    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:47.676618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.654991    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:31:47.678489 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.678455    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:47.697699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.696867    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:31:47.725700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.720269    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:47.725700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.720330    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:47.768428 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.768386    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:47.785746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:47.782033    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:31:48.018881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.018839    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:31:48.090701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.090325    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.380756 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.372778    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.404274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.393784    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:48.404274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.394197    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:31:48.491847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.491143    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:31:48.516139 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.512229    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:48.576752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.573782    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.599119 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.594783    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:31:48.646780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.646442    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:31:48.653818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.653744    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.655158 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.655113    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.698837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.698789    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.699512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.699080    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:48.700841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.700818    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:48.702511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:48.702488    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:49.028040 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.020511    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.108955 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.090771    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.423754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.417103    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.423754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.418526    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.423754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.418853    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.423754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.420640    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.625500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.625447    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.676754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.626835    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.677361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.677327    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.696123    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.699150    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.699194    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.699240    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.699866    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.701709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:49.700482    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.026809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.026766    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.087704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.086794    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:31:50.087704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.086869    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:31:50.122094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.095571    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.138712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.127901    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:50.209700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.173125    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\") pod \"4194440b-9220-4f02-b801-caabfe3b3034\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") "
Oct 30 19:31:50.367935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.367851    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\" (UniqueName: \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\") pod \"4194440b-9220-4f02-b801-caabfe3b3034\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") "
Oct 30 19:31:50.371697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.368246    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\") pod \"4194440b-9220-4f02-b801-caabfe3b3034\" (UID: \"4194440b-9220-4f02-b801-caabfe3b3034\") "
Oct 30 19:31:50.391699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.376928    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca" pod "4194440b-9220-4f02-b801-caabfe3b3034" (UID: "4194440b-9220-4f02-b801-caabfe3b3034"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:50.429703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.392834    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:50.448838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.430384    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:31:50.488087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.430422    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:50.489801 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.489763    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.572219 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.555427    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca" pod "4194440b-9220-4f02-b801-caabfe3b3034" (UID: "4194440b-9220-4f02-b801-caabfe3b3034"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:50.572219 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.558521    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:31:50.572219 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.560490    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4194440b-9220-4f02-b801-caabfe3b3034-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:50.628755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.594714    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.628755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.607030    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9" pod "4194440b-9220-4f02-b801-caabfe3b3034" (UID: "4194440b-9220-4f02-b801-caabfe3b3034"). InnerVolumeSpecName "kube-api-access-x4hn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:50.629189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.629153    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.635620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.632912    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:31:50.697702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.672996    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4hn9\" (UniqueName: \"kubernetes.io/projected/4194440b-9220-4f02-b801-caabfe3b3034-kube-api-access-x4hn9\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:50.697930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.683162    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.699282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.696805    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.705751 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.701092    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.706427 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.701122    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:31:50.729050 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.701143    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:50.729607 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.702389    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:50.729842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.729805    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:51.000701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.998719    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2" event={"ID":"4194440b-9220-4f02-b801-caabfe3b3034","Type":"ContainerDied","Data":"1d848dcd7ac2722fb04be6f0434cb9b4a17fff19fcffb79754d2e4714a7694b9"}
Oct 30 19:31:51.000701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.998788    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d848dcd7ac2722fb04be6f0434cb9b4a17fff19fcffb79754d2e4714a7694b9"
Oct 30 19:31:51.000701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:50.998864    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"
Oct 30 19:31:51.029747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.029705    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.099213 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.099172    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.156168 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.156124    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:51.156451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.156420    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.293887 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.293776    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.484729 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:31:51.473205    5033 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4194440b_9220_4f02_b801_caabfe3b3034.slice/crio-1d848dcd7ac2722fb04be6f0434cb9b4a17fff19fcffb79754d2e4714a7694b9\": RecentStats: unable to find data in memory cache]"
Oct 30 19:31:51.498971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.498886    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.499661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.499621    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.508071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.500100    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.565508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.565163    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.619698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.595401    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:51.646501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.601568    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.712183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.645052    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.759918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.759887    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:31:51.783894 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.657397    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:31:51.875481 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.875389    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:51.875764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.875728    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.918703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.891520    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.918703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.892228    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:51.918703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.892965    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.004722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.976484    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:52.004722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.976550    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:52.004722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.977762    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.004722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:51.979110    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.004722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.000763    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.052969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.050763    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.095586 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.095367    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.109744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.105436    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.323726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.311647    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/health\": context deadline exceeded"
Oct 30 19:31:52.530588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.528655    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.530588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.530108    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.530588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.530395    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.594962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.594740    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.666980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.666943    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:52.667258 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.667228    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:52.720437 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.720386    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.738695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.737598    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.884788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.854767    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.894814 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.894764    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.906503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.895013    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.906503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.895041    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:31:52.956036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.948104    5033 patch_prober.go:29] interesting pod/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:52.956036 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.948170    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:52.985702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:52.984778    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:31:53.054699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.025028    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:53.065874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.065830    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:53.144401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.144018    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:31:53.266823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.259076    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:31:53.325416 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.325379    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": context deadline exceeded"
Oct 30 19:31:53.347937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.347818    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:53.516550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.516506    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:31:53.516863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.516847    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"]
Oct 30 19:31:53.519342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.519323    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4194440b-9220-4f02-b801-caabfe3b3034" containerName="revoke-keys"
Oct 30 19:31:53.519483 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.519471    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4194440b-9220-4f02-b801-caabfe3b3034" containerName="revoke-keys"
Oct 30 19:31:53.519802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.519789    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4194440b-9220-4f02-b801-caabfe3b3034" containerName="revoke-keys"
Oct 30 19:31:53.545826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.539124    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:53.594738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.594709    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:31:53.594960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.594946    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"]
Oct 30 19:31:53.595098 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.595088    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:31:53.595208 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.595199    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:31:53.613435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.613411    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.622191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.622162    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:31:53.622396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.622380    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:31:53.632248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.631199    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:31:53.632248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.632085    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:31:53.666707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.666665    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:53.666889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.666865    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:53.720303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.720268    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.720471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.720322    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4n4k\" (UniqueName: \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.720538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.720507    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.732594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.724807    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:31:53.732594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.728451    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:31:53.732594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.729399    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:31:53.733213 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.733197    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:31:53.738582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.738554    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:53.747637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.743989    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:31:53.812742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.812644    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:31:53.814859 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.814837    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:31:53.821601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.821571    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.821755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.821637    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4n4k\" (UniqueName: \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.821863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.821846    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.824140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.824109    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.830562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.830540    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:31:53.832102 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.832074    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.836168 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.836142    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4n4k\" (UniqueName: \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\") pod \"maas-api-revoke-sub-6ec888fd-4n54m\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:53.953709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:53.953171    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:54.323225 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:54.323189    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode065156e_b44c_4892_a8c9_42e9a0a0efd7.slice/crio-3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8 WatchSource:0}: Error finding container 3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8: Status 404 returned error can't find the container with id 3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8
Oct 30 19:31:54.323613 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:54.323380    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"]
Oct 30 19:31:54.611936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:54.611872    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m" event={"ID":"e065156e-b44c-4892-a8c9-42e9a0a0efd7","Type":"ContainerStarted","Data":"3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8"}
Oct 30 19:31:54.689498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:54.688821    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:54.689498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:54.688891    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:55.621160 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:55.621127    5033 generic.go:356] "Generic (PLEG): container finished" podID="e065156e-b44c-4892-a8c9-42e9a0a0efd7" containerID="2f7b8e79a7de321e715c6706cb37f0a25cbf6630ed7cf91a613d795b0e6ca0e6" exitCode=0
Oct 30 19:31:55.621618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:55.621211    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m" event={"ID":"e065156e-b44c-4892-a8c9-42e9a0a0efd7","Type":"ContainerDied","Data":"2f7b8e79a7de321e715c6706cb37f0a25cbf6630ed7cf91a613d795b0e6ca0e6"}
Oct 30 19:31:55.680741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:55.679622    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:55.680741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:55.679696    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:56.666478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.666059    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 19:31:56.666478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.666115    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 19:31:56.876557 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.876527    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:56.933399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.933316    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\") pod \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") "
Oct 30 19:31:56.933552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.933533    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\") pod \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") "
Oct 30 19:31:56.933784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.933761    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\" (UniqueName: \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\") pod \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\" (UID: \"e065156e-b44c-4892-a8c9-42e9a0a0efd7\") "
Oct 30 19:31:56.933896 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.933812    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca" pod "e065156e-b44c-4892-a8c9-42e9a0a0efd7" (UID: "e065156e-b44c-4892-a8c9-42e9a0a0efd7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:56.934366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.934344    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:56.935839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.935815    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca" pod "e065156e-b44c-4892-a8c9-42e9a0a0efd7" (UID: "e065156e-b44c-4892-a8c9-42e9a0a0efd7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:56.936374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:56.936357    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k" pod "e065156e-b44c-4892-a8c9-42e9a0a0efd7" (UID: "e065156e-b44c-4892-a8c9-42e9a0a0efd7"). InnerVolumeSpecName "kube-api-access-m4n4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:57.036072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:57.036043    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e065156e-b44c-4892-a8c9-42e9a0a0efd7-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:57.036072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:57.036070    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4n4k\" (UniqueName: \"kubernetes.io/projected/e065156e-b44c-4892-a8c9-42e9a0a0efd7-kube-api-access-m4n4k\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:57.653882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:57.653856    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"
Oct 30 19:31:57.654074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:57.653891    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m" event={"ID":"e065156e-b44c-4892-a8c9-42e9a0a0efd7","Type":"ContainerDied","Data":"3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8"}
Oct 30 19:31:57.654074 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:57.653937    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3684b9696f15919df8bcafae74f9489535dd4aeb3a736ceb68f20616cb86e4d8"
Oct 30 19:31:58.162034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.162001    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:31:58.163108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.163082    5033 cpu_manager.go:414] "containerMap: removing container" podUID="e065156e-b44c-4892-a8c9-42e9a0a0efd7" containerName="revoke-keys"
Oct 30 19:31:58.163108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.163106    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e065156e-b44c-4892-a8c9-42e9a0a0efd7" containerName="revoke-keys"
Oct 30 19:31:58.165817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.163347    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="e065156e-b44c-4892-a8c9-42e9a0a0efd7" containerName="revoke-keys"
Oct 30 19:31:58.168378 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.168355    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.196664 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.196637    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:31:58.261046 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.261014    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.261313 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.261277    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.261519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.261498    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qksrb\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.261729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.261710    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.262114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.262092    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.262260 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.262243    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.262429 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.262393    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.262580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.262563    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.262741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.262721    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.364075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.364018    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.364364 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.364340    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.364519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.364502    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.364662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.364647    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.364935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.364904    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.372289    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.372565    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.372601    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.372632    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qksrb\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.372697    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.373251    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.374797    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.377816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.376244    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.380865 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.379014    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.403213 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.399220    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.403213 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.399904    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qksrb\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.413812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.404341    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.413812 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.410947    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\") pod \"e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") " pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.494165 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.493886    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:31:58.611175 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.611139    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"]
Oct 30 19:31:58.629726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.629694    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"]
Oct 30 19:31:58.629900 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.629792    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.642656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.641967    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:31:58.642656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.642603    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:31:58.668886 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.668823    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:58.692987 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.692957    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt4lw\" (UniqueName: \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.700063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.699810    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.700063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.699867    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.716501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.715197    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:58.716501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.715244    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:58.802904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.802866    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.803209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.803186    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt4lw\" (UniqueName: \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.803339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.803319    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.804940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.804909    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.812571 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.807836    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.819867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.819073    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt4lw\" (UniqueName: \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\") pod \"maas-api-revoke-sub-b314d387-jsptn\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:58.890164 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:58.890133    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6c61fe_0de4_4c23_8753_d1a0b016a9a4.slice/crio-52f48653646f1aded62a79d662fed884d18518837e07e8532b5b21c12419a822 WatchSource:0}: Error finding container 52f48653646f1aded62a79d662fed884d18518837e07e8532b5b21c12419a822: Status 404 returned error can't find the container with id 52f48653646f1aded62a79d662fed884d18518837e07e8532b5b21c12419a822
Oct 30 19:31:58.892300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.892271    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:31:58.893573 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.893538    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:58.893660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.893605    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:58.893660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.893638    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:31:58.948158 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.948133    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:58.994131 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:58.993864    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:31:59.316462 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:31:59.316425    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cc2627_ed53_406f_aafd_cf5fb847ee2b.slice/crio-b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610 WatchSource:0}: Error finding container b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610: Status 404 returned error can't find the container with id b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610
Oct 30 19:31:59.316835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.316601    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"]
Oct 30 19:31:59.682885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.682792    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn" event={"ID":"91cc2627-ed53-406f-aafd-cf5fb847ee2b","Type":"ContainerStarted","Data":"b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610"}
Oct 30 19:31:59.684967 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.684937    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" event={"ID":"db6c61fe-0de4-4c23-8753-d1a0b016a9a4","Type":"ContainerStarted","Data":"3aba93ac98bcc1a29bfa64e6e058cf6e90ae59bfb78f540b23fff95851d14954"}
Oct 30 19:31:59.685121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.684987    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" event={"ID":"db6c61fe-0de4-4c23-8753-d1a0b016a9a4","Type":"ContainerStarted","Data":"52f48653646f1aded62a79d662fed884d18518837e07e8532b5b21c12419a822"}
Oct 30 19:31:59.686282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.686250    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:31:59.715724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:31:59.710363    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podStartSLOduration=1.710350604 podStartE2EDuration="1.710350604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:59.704827317 +0000 UTC m=+784.674468153" watchObservedRunningTime="2026-10-30 19:31:59.710350604 +0000 UTC m=+784.679991503"
Oct 30 19:32:00.494846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:00.494590    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:32:00.496046 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:00.496014    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:00.496209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:00.496070    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:00.700382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:00.700345    5033 generic.go:356] "Generic (PLEG): container finished" podID="91cc2627-ed53-406f-aafd-cf5fb847ee2b" containerID="f1ac7797abb5fed9c64a5bb77a894ff03e2ddc19ef77adecb6c9cf22c51d1f81" exitCode=0
Oct 30 19:32:00.700712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:00.700454    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn" event={"ID":"91cc2627-ed53-406f-aafd-cf5fb847ee2b","Type":"ContainerDied","Data":"f1ac7797abb5fed9c64a5bb77a894ff03e2ddc19ef77adecb6c9cf22c51d1f81"}
Oct 30 19:32:01.511395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:01.508939    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:01.511395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:01.509027    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:01.939602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:01.939576    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:32:02.110699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.110595    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\") pod \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") "
Oct 30 19:32:02.110699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.110645    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\" (UniqueName: \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\") pod \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") "
Oct 30 19:32:02.110935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.110916    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\") pod \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\" (UID: \"91cc2627-ed53-406f-aafd-cf5fb847ee2b\") "
Oct 30 19:32:02.111235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.111208    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca" pod "91cc2627-ed53-406f-aafd-cf5fb847ee2b" (UID: "91cc2627-ed53-406f-aafd-cf5fb847ee2b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:02.111717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.111696    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:02.113058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.113030    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca" pod "91cc2627-ed53-406f-aafd-cf5fb847ee2b" (UID: "91cc2627-ed53-406f-aafd-cf5fb847ee2b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:02.113176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.113155    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw" pod "91cc2627-ed53-406f-aafd-cf5fb847ee2b" (UID: "91cc2627-ed53-406f-aafd-cf5fb847ee2b"). InnerVolumeSpecName "kube-api-access-tt4lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:02.214044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.214011    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91cc2627-ed53-406f-aafd-cf5fb847ee2b-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:02.214044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.214042    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt4lw\" (UniqueName: \"kubernetes.io/projected/91cc2627-ed53-406f-aafd-cf5fb847ee2b-kube-api-access-tt4lw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:02.542714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.530351    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:02.542714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.530409    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:02.724077 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.724047    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"
Oct 30 19:32:02.724235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.724092    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn" event={"ID":"91cc2627-ed53-406f-aafd-cf5fb847ee2b","Type":"ContainerDied","Data":"b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610"}
Oct 30 19:32:02.724235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:02.724142    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b30e0642c00e56eb128aea2f02834beb58f88906cfeca579cfa08a41028c9610"
Oct 30 19:32:03.495029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:03.494986    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:03.495318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:03.495286    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:04.495034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:04.494991    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:04.495553 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:04.495046    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:05.008129 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:05.008095    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:32:05.494939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:05.494899    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:32:05.495134 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:05.494963    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:32:06.190717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.190660    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:32:06.212479 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.212441    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:32:06.225605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.224754    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:32:06.727720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.726111    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:32:06.727720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.726885    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:32:06.728250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:06.728121    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:32:09.602824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.602775    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:09.661427 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.661382    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:09.662083 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.661700    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:09.682002 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.681968    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:09.706664 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.706625    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:09.730336 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:09.730296    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.025843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.025800    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.105808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.105757    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.384862 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.384759    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.436441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.436394    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.736701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.734841    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.736701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.735620    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:32:10.736701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.735912    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.754788    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.755669    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.755987    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.756251    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.756533    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.763295    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.763349    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.763382    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.764137    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.773180    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.773859    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:32:10.803700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.775097    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:10.878698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:10.878259    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:11.038868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.038774    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:11.039941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.039918    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:32:11.490802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.451180    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.621740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.596025    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.623164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.623120    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.774777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.761438    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.774777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.764345    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.774777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.765160    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.774777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.765388    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.792900 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.792853    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.825826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.825781    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:11.904728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.862832    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Oct 30 19:32:12.057529 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:11.896984    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.234702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.023054    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:32:12.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.262832    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:32:12.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.263019    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:32:12.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.082783    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:12.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.229981    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.316902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.306615    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.394695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.233689    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:32:12.479516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.471227    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.652128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.651937    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.653796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.653775    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:32:12.748839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.748797    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.794019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.793968    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.805947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.805910    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.866702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.836988    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.892873 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.848769    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.978538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.848966    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:32:12.978538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.848992    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:12.978538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.849011    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.066116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:12.893957    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:32:13.084837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.084780    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.115476 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.115428    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.146772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.146733    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:13.336016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.316813    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.344799 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.344759    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.395700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.383268    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.499935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.499892    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.502536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.502498    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.733702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.733491    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.753700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.750119    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.811697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.810986    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.832700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.812317    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.883881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.866503    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:32:13.884306 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.884280    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:32:13.919702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.884150    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:32:13.961918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.946981    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.961918 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.956906    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.967101 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.967059    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.968498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.968465    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:13.987755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:13.984762    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:32:14.013717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.011857    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.184909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.135596    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:32:14.343750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.343644    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:14.364301 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.364243    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.405950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.405909    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.529696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.503775    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.742487 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.742443    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.768698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.751256    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.839739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.839244    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.839739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.839628    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.864713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.846334    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:32:14.926699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.926178    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:32:14.957702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.952093    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.978700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.965762    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.978700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.968603    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:14.978700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:14.968740    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:32:15.209714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.206943    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:32:15.226709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.225794    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:15.307698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.275449    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:32:15.307698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.291239    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:32:15.307698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.294920    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:32:15.353262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.353222    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:15.546498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.546394    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:32:15.574379 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.574339    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": context deadline exceeded"
Oct 30 19:32:15.611013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.610979    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:32:15.611343 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.611322    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:32:15.611525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.611512    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:15.611834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.611818    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:32:15.625288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.624844    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:32:15.631695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.629957    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:15.640508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.640053    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:32:15.642887 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.642848    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:32:15.644394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.644378    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:32:15.683810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.683181    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:32:15.796843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.796760    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"]
Oct 30 19:32:15.797807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.797778    5033 cpu_manager.go:414] "containerMap: removing container" podUID="91cc2627-ed53-406f-aafd-cf5fb847ee2b" containerName="revoke-keys"
Oct 30 19:32:15.797807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.797808    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91cc2627-ed53-406f-aafd-cf5fb847ee2b" containerName="revoke-keys"
Oct 30 19:32:15.798197 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.798024    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="91cc2627-ed53-406f-aafd-cf5fb847ee2b" containerName="revoke-keys"
Oct 30 19:32:15.805501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.803359    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:15.809976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.808604    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:15.809976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.808780    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:32:15.811965 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.811882    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"]
Oct 30 19:32:15.826995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.823500    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"]
Oct 30 19:32:15.832885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.829378    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:15.849976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.849098    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c94mw\" (UniqueName: \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:15.849976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.849283    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:15.849976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.849312    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:15.849976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.849941    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"]
Oct 30 19:32:15.931283 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.930784    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:32:15.993286 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:15.950596    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.003707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.001932    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c94mw\" (UniqueName: \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.003707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.002002    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.003707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.002094    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc9dx\" (UniqueName: \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.003707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.002253    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.003707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.002282    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.004094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.003795    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.020204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.019631    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c94mw\" (UniqueName: \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.022889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.022864    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38a69b09-n2vvc\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.103741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.103639    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.104085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.104057    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.104753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.104732    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc9dx\" (UniqueName: \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.105466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.105438    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.132382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.132205    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc9dx\" (UniqueName: \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.142300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.142274    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d43671e5-z9h56\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.179148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.179117    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:16.221231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.221193    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:16.534310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.534271    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"]
Oct 30 19:32:16.539171 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:16.539133    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52be9153_2a15_456c_a2ed_ae191b30ca25.slice/crio-77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3 WatchSource:0}: Error finding container 77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3: Status 404 returned error can't find the container with id 77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3
Oct 30 19:32:16.569389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:16.567141    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"]
Oct 30 19:32:16.577041 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:16.577016    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c84868_8e99_419e_96eb_31980f08f5ac.slice/crio-d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84 WatchSource:0}: Error finding container d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84: Status 404 returned error can't find the container with id d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84
Oct 30 19:32:17.021829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.019181    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56" event={"ID":"d7c84868-8e99-419e-96eb-31980f08f5ac","Type":"ContainerStarted","Data":"556dbef4bd84efa89bd942005e08dcb2c92cb00911a86d8524e9faaacc856e38"}
Oct 30 19:32:17.021829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.019242    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56" event={"ID":"d7c84868-8e99-419e-96eb-31980f08f5ac","Type":"ContainerStarted","Data":"d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84"}
Oct 30 19:32:17.021829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.021062    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc" event={"ID":"52be9153-2a15-456c-a2ed-ae191b30ca25","Type":"ContainerStarted","Data":"aa5b5833887b5ba76103ad4ac054184bcf32b923ed38564aa9134331e47ba2f2"}
Oct 30 19:32:17.021829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.021102    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc" event={"ID":"52be9153-2a15-456c-a2ed-ae191b30ca25","Type":"ContainerStarted","Data":"77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3"}
Oct 30 19:32:17.046095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.046050    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56" podStartSLOduration=2.046037535 podStartE2EDuration="2.046037535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:17.04136126 +0000 UTC m=+802.011002097" watchObservedRunningTime="2026-10-30 19:32:17.046037535 +0000 UTC m=+802.015678370"
Oct 30 19:32:17.082057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:17.082013    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc" podStartSLOduration=2.081999964 podStartE2EDuration="2.081999964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:17.07736651 +0000 UTC m=+802.047007345" watchObservedRunningTime="2026-10-30 19:32:17.081999964 +0000 UTC m=+802.051640799"
Oct 30 19:32:18.030792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:18.030755    5033 generic.go:356] "Generic (PLEG): container finished" podID="d7c84868-8e99-419e-96eb-31980f08f5ac" containerID="556dbef4bd84efa89bd942005e08dcb2c92cb00911a86d8524e9faaacc856e38" exitCode=0
Oct 30 19:32:18.031249 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:18.030858    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56" event={"ID":"d7c84868-8e99-419e-96eb-31980f08f5ac","Type":"ContainerDied","Data":"556dbef4bd84efa89bd942005e08dcb2c92cb00911a86d8524e9faaacc856e38"}
Oct 30 19:32:18.032597 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:18.032573    5033 generic.go:356] "Generic (PLEG): container finished" podID="52be9153-2a15-456c-a2ed-ae191b30ca25" containerID="aa5b5833887b5ba76103ad4ac054184bcf32b923ed38564aa9134331e47ba2f2" exitCode=0
Oct 30 19:32:18.032727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:18.032600    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc" event={"ID":"52be9153-2a15-456c-a2ed-ae191b30ca25","Type":"ContainerDied","Data":"aa5b5833887b5ba76103ad4ac054184bcf32b923ed38564aa9134331e47ba2f2"}
Oct 30 19:32:19.303222 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.303166    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:19.307428 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.307391    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:19.361973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.361938    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\") pod \"52be9153-2a15-456c-a2ed-ae191b30ca25\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") "
Oct 30 19:32:19.362131 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.362119    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\" (UniqueName: \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\") pod \"d7c84868-8e99-419e-96eb-31980f08f5ac\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") "
Oct 30 19:32:19.362268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.362251    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\") pod \"d7c84868-8e99-419e-96eb-31980f08f5ac\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") "
Oct 30 19:32:19.362343 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.362317    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\") pod \"52be9153-2a15-456c-a2ed-ae191b30ca25\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") "
Oct 30 19:32:19.362394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.362356    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\") pod \"d7c84868-8e99-419e-96eb-31980f08f5ac\" (UID: \"d7c84868-8e99-419e-96eb-31980f08f5ac\") "
Oct 30 19:32:19.362489 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.362466    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\" (UniqueName: \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\") pod \"52be9153-2a15-456c-a2ed-ae191b30ca25\" (UID: \"52be9153-2a15-456c-a2ed-ae191b30ca25\") "
Oct 30 19:32:19.363129 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.363100    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca" pod "d7c84868-8e99-419e-96eb-31980f08f5ac" (UID: "d7c84868-8e99-419e-96eb-31980f08f5ac"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:19.363788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.363484    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca" pod "52be9153-2a15-456c-a2ed-ae191b30ca25" (UID: "52be9153-2a15-456c-a2ed-ae191b30ca25"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:19.372963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.365305    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.372963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.365334    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.372963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.371853    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx" pod "d7c84868-8e99-419e-96eb-31980f08f5ac" (UID: "d7c84868-8e99-419e-96eb-31980f08f5ac"). InnerVolumeSpecName "kube-api-access-jc9dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:19.373914 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.373882    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw" pod "52be9153-2a15-456c-a2ed-ae191b30ca25" (UID: "52be9153-2a15-456c-a2ed-ae191b30ca25"). InnerVolumeSpecName "kube-api-access-c94mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:19.374087 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.374071    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca" pod "d7c84868-8e99-419e-96eb-31980f08f5ac" (UID: "d7c84868-8e99-419e-96eb-31980f08f5ac"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:19.375816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.375785    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca" pod "52be9153-2a15-456c-a2ed-ae191b30ca25" (UID: "52be9153-2a15-456c-a2ed-ae191b30ca25"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:19.467655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.467617    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7c84868-8e99-419e-96eb-31980f08f5ac-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.467797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.467658    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52be9153-2a15-456c-a2ed-ae191b30ca25-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.467797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.467719    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c94mw\" (UniqueName: \"kubernetes.io/projected/52be9153-2a15-456c-a2ed-ae191b30ca25-kube-api-access-c94mw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.467797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.467737    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc9dx\" (UniqueName: \"kubernetes.io/projected/d7c84868-8e99-419e-96eb-31980f08f5ac-kube-api-access-jc9dx\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:19.706724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.706657    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"]
Oct 30 19:32:19.707718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.707669    5033 cpu_manager.go:414] "containerMap: removing container" podUID="d7c84868-8e99-419e-96eb-31980f08f5ac" containerName="revoke-keys"
Oct 30 19:32:19.707718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.707707    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c84868-8e99-419e-96eb-31980f08f5ac" containerName="revoke-keys"
Oct 30 19:32:19.707910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.707789    5033 cpu_manager.go:414] "containerMap: removing container" podUID="52be9153-2a15-456c-a2ed-ae191b30ca25" containerName="revoke-keys"
Oct 30 19:32:19.707910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.707798    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52be9153-2a15-456c-a2ed-ae191b30ca25" containerName="revoke-keys"
Oct 30 19:32:19.708016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.708002    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="52be9153-2a15-456c-a2ed-ae191b30ca25" containerName="revoke-keys"
Oct 30 19:32:19.708068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.708021    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c84868-8e99-419e-96eb-31980f08f5ac" containerName="revoke-keys"
Oct 30 19:32:19.712870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.712847    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.713004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.712899    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"]
Oct 30 19:32:19.717646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.717621    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.718626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.718602    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"]
Oct 30 19:32:19.723170 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.723147    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"]
Oct 30 19:32:19.772367 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.772330    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7c6\" (UniqueName: \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.774098 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.774069    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.774300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.774281    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.774433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.774419    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.774524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.774511    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rmjz\" (UniqueName: \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.774623 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.774608    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.847038 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.846966    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing" containerID="cri-o://eeb5398f42908c8f2403175aa7f6ef1bc683aa3215ca294c0ba2853cd0a405d6" gracePeriod=25
Oct 30 19:32:19.876959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.876925    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.877085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.876985    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.877085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.877027    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.877085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.877053    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rmjz\" (UniqueName: \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.877085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.877078    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.877295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.877272    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7c6\" (UniqueName: \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.878056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.878022    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.878297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.878263    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.880546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.880519    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.885009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.884981    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:19.887935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.887907    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7c6\" (UniqueName: \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\") pod \"maas-api-revoke-sub-75e08bdd-kqfxw\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:19.888555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:19.888535    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rmjz\" (UniqueName: \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\") pod \"maas-api-revoke-sub-2522e5ee-zzf7q\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:20.052770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.049470    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:20.059897 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.059389    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:20.067995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.062884    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56" event={"ID":"d7c84868-8e99-419e-96eb-31980f08f5ac","Type":"ContainerDied","Data":"d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84"}
Oct 30 19:32:20.067995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.062977    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d32208dda74473b17e29f78ed1459efe1ffc4e3d37a7040358ba13fbb1a53f84"
Oct 30 19:32:20.067995 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.063060    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"
Oct 30 19:32:20.072725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.070538    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc" event={"ID":"52be9153-2a15-456c-a2ed-ae191b30ca25","Type":"ContainerDied","Data":"77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3"}
Oct 30 19:32:20.072725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.070593    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77f98bbd7bc9ae3185a0f7f243c0c9439f3374051f086a19ddd00799aa9c47d3"
Oct 30 19:32:20.072725 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.070758    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"
Oct 30 19:32:20.401013 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:20.400981    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06cc768f_89b3_4420_8664_e3f65b6ef227.slice/crio-e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c WatchSource:0}: Error finding container e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c: Status 404 returned error can't find the container with id e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c
Oct 30 19:32:20.402371 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.402344    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"]
Oct 30 19:32:20.420796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:20.420766    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"]
Oct 30 19:32:20.424905 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:20.424880    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5afd0952_f8e4_4de3_975d_225a74f220f1.slice/crio-609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c WatchSource:0}: Error finding container 609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c: Status 404 returned error can't find the container with id 609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c
Oct 30 19:32:21.082980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.082938    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q" event={"ID":"06cc768f-89b3-4420-8664-e3f65b6ef227","Type":"ContainerStarted","Data":"85a0fa88e08f6d9a6f3f52382547bf4dccf13a44112fbbcd4cd78cb113b6cb98"}
Oct 30 19:32:21.083159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.083010    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q" event={"ID":"06cc768f-89b3-4420-8664-e3f65b6ef227","Type":"ContainerStarted","Data":"e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c"}
Oct 30 19:32:21.084830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.084800    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw" event={"ID":"5afd0952-f8e4-4de3-975d-225a74f220f1","Type":"ContainerStarted","Data":"40008faa3158458be2e923e54c7b282cc0d01585fb3227c779ffed15cdf755c0"}
Oct 30 19:32:21.084960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.084850    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw" event={"ID":"5afd0952-f8e4-4de3-975d-225a74f220f1","Type":"ContainerStarted","Data":"609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c"}
Oct 30 19:32:21.103219 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.103174    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q" podStartSLOduration=2.103161024 podStartE2EDuration="2.103161024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:21.101904487 +0000 UTC m=+806.071545315" watchObservedRunningTime="2026-10-30 19:32:21.103161024 +0000 UTC m=+806.072801877"
Oct 30 19:32:21.125864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.125818    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw" podStartSLOduration=2.125807296 podStartE2EDuration="2.125807296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:21.119936528 +0000 UTC m=+806.089577378" watchObservedRunningTime="2026-10-30 19:32:21.125807296 +0000 UTC m=+806.095448142"
Oct 30 19:32:21.221450 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.221405    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" containerID="cri-o://1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7" gracePeriod=45
Oct 30 19:32:21.240775 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.240719    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" containerID="cri-o://47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347" gracePeriod=45
Oct 30 19:32:21.252945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.252914    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" containerID="cri-o://bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53" gracePeriod=45
Oct 30 19:32:21.488765 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.488626    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": dial tcp 10.132.0.70:8000: connect: connection refused"
Oct 30 19:32:21.730802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.730770    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:32:21.733241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.733213    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:32:21.739941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.739833    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:32:21.837889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.837852    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\" (UniqueName: \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.839404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.839373    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\" (UniqueName: \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.858830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.858797    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.859051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859031    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.859176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859161    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.859293 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859279    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\" (UniqueName: \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.859443 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859428    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.859556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859542    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\" (UniqueName: \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.859670 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.859656    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\" (UniqueName: \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.862383 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.862356    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.862628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.862609    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.862816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.862799    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.875806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.875777    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.876107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.876088    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.876223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.876208    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\") pod \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\" (UID: \"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166\") "
Oct 30 19:32:21.876427 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.876407    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\") pod \"3feecb03-3ef3-4546-876d-7b6d079a202c\" (UID: \"3feecb03-3ef3-4546-876d-7b6d079a202c\") "
Oct 30 19:32:21.876611 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.876593    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.876792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.876776    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\") pod \"4e423276-0c16-418c-bbd3-5cc1a698cd98\" (UID: \"4e423276-0c16-418c-bbd3-5cc1a698cd98\") "
Oct 30 19:32:21.911577 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.911538    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.915334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.915048    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.917720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.917427    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.918064 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.917882    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.918518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.918213    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.921136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.920808    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "kube-api-access-qlvwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.921560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.921287    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.937754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.923455    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.938361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.938147    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.938779 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.938528    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.939259 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.938970    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.941786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.940153    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.948864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.948815    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "kube-api-access-rbvqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.952775 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.952739    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.953046 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.953016    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "kube-api-access-nxxkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.956458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.956424    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs" pod "4e423276-0c16-418c-bbd3-5cc1a698cd98" (UID: "4e423276-0c16-418c-bbd3-5cc1a698cd98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.956703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.956658    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm" pod "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" (UID: "c4418b18-b0ad-44cf-b5f8-2f0a2afeb166"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.980800 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.980762    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home" pod "3feecb03-3ef3-4546-876d-7b6d079a202c" (UID: "3feecb03-3ef3-4546-876d-7b6d079a202c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.992834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.991327    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993485 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993463    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbvqj\" (UniqueName: \"kubernetes.io/projected/3feecb03-3ef3-4546-876d-7b6d079a202c-kube-api-access-rbvqj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993603    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxxkg\" (UniqueName: \"kubernetes.io/projected/4e423276-0c16-418c-bbd3-5cc1a698cd98-kube-api-access-nxxkg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993731 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993719    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993808    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993881    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.993960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.993951    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.994117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.994105    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.994192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.994182    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.995536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.995505    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.995741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.995713    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.995893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.995871    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e423276-0c16-418c-bbd3-5cc1a698cd98-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996019 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.995999    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996164 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.996144    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3feecb03-3ef3-4546-876d-7b6d079a202c-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.996283    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.996393    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e423276-0c16-418c-bbd3-5cc1a698cd98-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.996468    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3feecb03-3ef3-4546-876d-7b6d079a202c-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.996563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:21.996550    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlvwz\" (UniqueName: \"kubernetes.io/projected/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166-kube-api-access-qlvwz\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:22.113696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.109960    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:22.117473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.117440    5033 generic.go:356] "Generic (PLEG): container finished" podID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerID="1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7" exitCode=0
Oct 30 19:32:22.117743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.117704    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" event={"ID":"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166","Type":"ContainerDied","Data":"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"}
Oct 30 19:32:22.118665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.118625    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" event={"ID":"c4418b18-b0ad-44cf-b5f8-2f0a2afeb166","Type":"ContainerDied","Data":"a9c1b9c4a7fb9d7379a33f6d07208dc117aad510103d913cf7191627bbd15ff3"}
Oct 30 19:32:22.118665 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.118705    5033 scope.go:122] "RemoveContainer" containerID="1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"
Oct 30 19:32:22.118923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.118906    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"
Oct 30 19:32:22.128020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.127989    5033 generic.go:356] "Generic (PLEG): container finished" podID="06cc768f-89b3-4420-8664-e3f65b6ef227" containerID="85a0fa88e08f6d9a6f3f52382547bf4dccf13a44112fbbcd4cd78cb113b6cb98" exitCode=0
Oct 30 19:32:22.128155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.128123    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q" event={"ID":"06cc768f-89b3-4420-8664-e3f65b6ef227","Type":"ContainerDied","Data":"85a0fa88e08f6d9a6f3f52382547bf4dccf13a44112fbbcd4cd78cb113b6cb98"}
Oct 30 19:32:22.130559 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.130536    5033 generic.go:356] "Generic (PLEG): container finished" podID="5afd0952-f8e4-4de3-975d-225a74f220f1" containerID="40008faa3158458be2e923e54c7b282cc0d01585fb3227c779ffed15cdf755c0" exitCode=0
Oct 30 19:32:22.130652 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.130637    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw" event={"ID":"5afd0952-f8e4-4de3-975d-225a74f220f1","Type":"ContainerDied","Data":"40008faa3158458be2e923e54c7b282cc0d01585fb3227c779ffed15cdf755c0"}
Oct 30 19:32:22.132415 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.132393    5033 generic.go:356] "Generic (PLEG): container finished" podID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerID="47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347" exitCode=0
Oct 30 19:32:22.132541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.132520    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"
Oct 30 19:32:22.132746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.132587    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" event={"ID":"3feecb03-3ef3-4546-876d-7b6d079a202c","Type":"ContainerDied","Data":"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"}
Oct 30 19:32:22.132746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.132621    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd" event={"ID":"3feecb03-3ef3-4546-876d-7b6d079a202c","Type":"ContainerDied","Data":"2668583053b3543e03729f319edf03073b331fdcf6a219f1faab7d85c6aa9eff"}
Oct 30 19:32:22.134151 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.134127    5033 generic.go:356] "Generic (PLEG): container finished" podID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerID="bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53" exitCode=0
Oct 30 19:32:22.134253 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.134165    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" event={"ID":"4e423276-0c16-418c-bbd3-5cc1a698cd98","Type":"ContainerDied","Data":"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"}
Oct 30 19:32:22.134253 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.134191    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" event={"ID":"4e423276-0c16-418c-bbd3-5cc1a698cd98","Type":"ContainerDied","Data":"ba878bc2fdb52ac36139c44d49d8ac24a58a9613a5c6966f1defd71a40a69c06"}
Oct 30 19:32:22.134384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.134257    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"
Oct 30 19:32:22.139414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.139395    5033 scope.go:122] "RemoveContainer" containerID="1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"
Oct 30 19:32:22.139671 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:22.139649    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7\": container with ID starting with 1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7 not found: ID does not exist" containerID="1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"
Oct 30 19:32:22.139671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.139694    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7"} err="failed to get container status \"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7\": rpc error: code = NotFound desc = could not find container \"1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7\": container with ID starting with 1902c32c39323e1a38263027c4a94ebdadb13ff246d9597d0dac2ae60f50d8a7 not found: ID does not exist"
Oct 30 19:32:22.139830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.139719    5033 scope.go:122] "RemoveContainer" containerID="47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"
Oct 30 19:32:22.153126 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.153104    5033 scope.go:122] "RemoveContainer" containerID="47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"
Oct 30 19:32:22.153448 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:22.153422    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347\": container with ID starting with 47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347 not found: ID does not exist" containerID="47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"
Oct 30 19:32:22.153585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.153457    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347"} err="failed to get container status \"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347\": rpc error: code = NotFound desc = could not find container \"47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347\": container with ID starting with 47ac62920d32c5f95a1797afa9f4ad02d255e6cf00dc0cf26d32f17292b39347 not found: ID does not exist"
Oct 30 19:32:22.153585 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.153480    5033 scope.go:122] "RemoveContainer" containerID="bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"
Oct 30 19:32:22.168749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.168725    5033 scope.go:122] "RemoveContainer" containerID="bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"
Oct 30 19:32:22.169417 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:22.169388    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53\": container with ID starting with bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53 not found: ID does not exist" containerID="bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"
Oct 30 19:32:22.169514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.169432    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53"} err="failed to get container status \"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53\": rpc error: code = NotFound desc = could not find container \"bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53\": container with ID starting with bd1f1065f9b4c60d7e9fb78d9fd13518cdd9b94caf350f9d42512957e30d5d53 not found: ID does not exist"
Oct 30 19:32:22.206220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.206184    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:32:22.210926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.210896    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn"]
Oct 30 19:32:22.216495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.216467    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:32:22.220958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.220935    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/premium-w3-9f2d57-kserve-7c747f6f96-fpbtd"]
Oct 30 19:32:22.236152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.236129    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:32:22.240964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.240941    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr"]
Oct 30 19:32:22.633354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.633264    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/distinct-2-w3-9f2d57-kserve-54c4976b48-6tfwn" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:22.655809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:22.655778    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9f2d57/simulator-w3-9f2d57-kserve-78bb564b49-9plzr" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:23.174331 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.174298    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" path="/var/lib/kubelet/pods/3feecb03-3ef3-4546-876d-7b6d079a202c/volumes"
Oct 30 19:32:23.174889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.174873    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" path="/var/lib/kubelet/pods/4e423276-0c16-418c-bbd3-5cc1a698cd98/volumes"
Oct 30 19:32:23.175334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.175320    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" path="/var/lib/kubelet/pods/c4418b18-b0ad-44cf-b5f8-2f0a2afeb166/volumes"
Oct 30 19:32:23.207266 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.207221    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"]
Oct 30 19:32:23.210600 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.210575    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main"
Oct 30 19:32:23.210790 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.210775    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main"
Oct 30 19:32:23.210901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.210890    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main"
Oct 30 19:32:23.210990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.210979    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main"
Oct 30 19:32:23.211086 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.211076    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main"
Oct 30 19:32:23.211176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.211163    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main"
Oct 30 19:32:23.211516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.211499    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e423276-0c16-418c-bbd3-5cc1a698cd98" containerName="main"
Oct 30 19:32:23.215615 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.211624    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3feecb03-3ef3-4546-876d-7b6d079a202c" containerName="main"
Oct 30 19:32:23.215782 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.215766    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4418b18-b0ad-44cf-b5f8-2f0a2afeb166" containerName="main"
Oct 30 19:32:23.226839 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.226626    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.235018 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.234979    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"]
Oct 30 19:32:23.332928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.332889    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.333176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.333076    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.333356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.333333    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll28l\" (UniqueName: \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.403014 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.402940    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:23.406867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.406845    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:23.434787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.434763    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll28l\" (UniqueName: \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.434899 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.434859    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.434975 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.434905    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.435780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.435753    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.437227 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.437206    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.454288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.454266    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll28l\" (UniqueName: \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\") pod \"maas-api-revoke-sub-3ff1d931-znzh4\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.536495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536465    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\" (UniqueName: \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\") pod \"5afd0952-f8e4-4de3-975d-225a74f220f1\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") "
Oct 30 19:32:23.536770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536510    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\") pod \"06cc768f-89b3-4420-8664-e3f65b6ef227\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") "
Oct 30 19:32:23.536770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536610    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\") pod \"06cc768f-89b3-4420-8664-e3f65b6ef227\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") "
Oct 30 19:32:23.536974 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536789    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\") pod \"5afd0952-f8e4-4de3-975d-225a74f220f1\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") "
Oct 30 19:32:23.537057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536969    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\") pod \"5afd0952-f8e4-4de3-975d-225a74f220f1\" (UID: \"5afd0952-f8e4-4de3-975d-225a74f220f1\") "
Oct 30 19:32:23.537057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.536999    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\" (UniqueName: \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\") pod \"06cc768f-89b3-4420-8664-e3f65b6ef227\" (UID: \"06cc768f-89b3-4420-8664-e3f65b6ef227\") "
Oct 30 19:32:23.537215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.537159    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca" pod "06cc768f-89b3-4420-8664-e3f65b6ef227" (UID: "06cc768f-89b3-4420-8664-e3f65b6ef227"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:23.537594 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.537570    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca" pod "5afd0952-f8e4-4de3-975d-225a74f220f1" (UID: "5afd0952-f8e4-4de3-975d-225a74f220f1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:23.537833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.537808    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.537833 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.537833    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.539231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.539207    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6" pod "5afd0952-f8e4-4de3-975d-225a74f220f1" (UID: "5afd0952-f8e4-4de3-975d-225a74f220f1"). InnerVolumeSpecName "kube-api-access-9g7c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:23.539363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.539341    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz" pod "06cc768f-89b3-4420-8664-e3f65b6ef227" (UID: "06cc768f-89b3-4420-8664-e3f65b6ef227"). InnerVolumeSpecName "kube-api-access-9rmjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:23.539466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.539444    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca" pod "5afd0952-f8e4-4de3-975d-225a74f220f1" (UID: "5afd0952-f8e4-4de3-975d-225a74f220f1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:23.539730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.539617    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca" pod "06cc768f-89b3-4420-8664-e3f65b6ef227" (UID: "06cc768f-89b3-4420-8664-e3f65b6ef227"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:23.555370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.555345    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:23.646138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.646013    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g7c6\" (UniqueName: \"kubernetes.io/projected/5afd0952-f8e4-4de3-975d-225a74f220f1-kube-api-access-9g7c6\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.646138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.646051    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06cc768f-89b3-4420-8664-e3f65b6ef227-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.646138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.646076    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5afd0952-f8e4-4de3-975d-225a74f220f1-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.646138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.646091    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rmjz\" (UniqueName: \"kubernetes.io/projected/06cc768f-89b3-4420-8664-e3f65b6ef227-kube-api-access-9rmjz\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:23.746784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.746736    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"]
Oct 30 19:32:23.747756 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747735    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5afd0952-f8e4-4de3-975d-225a74f220f1" containerName="revoke-keys"
Oct 30 19:32:23.747883 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747760    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5afd0952-f8e4-4de3-975d-225a74f220f1" containerName="revoke-keys"
Oct 30 19:32:23.747883 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747796    5033 cpu_manager.go:414] "containerMap: removing container" podUID="06cc768f-89b3-4420-8664-e3f65b6ef227" containerName="revoke-keys"
Oct 30 19:32:23.747883 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747804    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06cc768f-89b3-4420-8664-e3f65b6ef227" containerName="revoke-keys"
Oct 30 19:32:23.747975 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747968    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="06cc768f-89b3-4420-8664-e3f65b6ef227" containerName="revoke-keys"
Oct 30 19:32:23.748008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.747988    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5afd0952-f8e4-4de3-975d-225a74f220f1" containerName="revoke-keys"
Oct 30 19:32:23.752855 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.752826    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.794376 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.794344    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"]
Oct 30 19:32:23.848724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.848692    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.848872 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.848850    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.849015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.849003    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26chw\" (UniqueName: \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.857847 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.857798    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"]
Oct 30 19:32:23.858295 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:23.858269    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0273b89_0592_4099_ad0a_93e4c5e4b0ad.slice/crio-7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb WatchSource:0}: Error finding container 7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb: Status 404 returned error can't find the container with id 7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb
Oct 30 19:32:23.950968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.950934    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.951220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.951197    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26chw\" (UniqueName: \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.951878 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.951856    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.952557 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.952064    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.956039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.956014    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:23.961564 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:23.961539    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26chw\" (UniqueName: \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\") pod \"maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:24.074501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.074434    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:24.163273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.163251    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"
Oct 30 19:32:24.163567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.163250    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q" event={"ID":"06cc768f-89b3-4420-8664-e3f65b6ef227","Type":"ContainerDied","Data":"e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c"}
Oct 30 19:32:24.163567 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.163539    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e62f7cfb5e16b5230d8f7b7afadda517b3abcd1290f1313c7d93015608ed4b9c"
Oct 30 19:32:24.169396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.166460    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw" event={"ID":"5afd0952-f8e4-4de3-975d-225a74f220f1","Type":"ContainerDied","Data":"609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c"}
Oct 30 19:32:24.169396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.166504    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="609a145eae9cb73560a8b0e24662d938834b2da5b1eb6047dd48abe839f7a45c"
Oct 30 19:32:24.169396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.166571    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"
Oct 30 19:32:24.179484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.179461    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4" event={"ID":"f0273b89-0592-4099-ad0a-93e4c5e4b0ad","Type":"ContainerStarted","Data":"7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb"}
Oct 30 19:32:24.365287 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:24.365248    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39c92d4_23f7_4ef5_abe1_2b083a360072.slice/crio-55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb WatchSource:0}: Error finding container 55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb: Status 404 returned error can't find the container with id 55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb
Oct 30 19:32:24.367612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:24.367491    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"]
Oct 30 19:32:25.196915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:25.196826    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn" event={"ID":"a39c92d4-23f7-4ef5-abe1-2b083a360072","Type":"ContainerStarted","Data":"10b51867b409c47731ccebe67bcd03b4512cce20b01a1b1d6a6d610daa8ce6a2"}
Oct 30 19:32:25.196915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:25.196908    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn" event={"ID":"a39c92d4-23f7-4ef5-abe1-2b083a360072","Type":"ContainerStarted","Data":"55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb"}
Oct 30 19:32:25.198511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:25.198483    5033 generic.go:356] "Generic (PLEG): container finished" podID="f0273b89-0592-4099-ad0a-93e4c5e4b0ad" containerID="adb53bedac2d07a40ddb53bde6d0f2b994c17a7f9f80a0631c4d7a85c4fd8f7d" exitCode=0
Oct 30 19:32:25.198625 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:25.198562    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4" event={"ID":"f0273b89-0592-4099-ad0a-93e4c5e4b0ad","Type":"ContainerDied","Data":"adb53bedac2d07a40ddb53bde6d0f2b994c17a7f9f80a0631c4d7a85c4fd8f7d"}
Oct 30 19:32:25.220117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:25.220070    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn" podStartSLOduration=2.220053124 podStartE2EDuration="2.220053124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:25.21409537 +0000 UTC m=+810.183736205" watchObservedRunningTime="2026-10-30 19:32:25.220053124 +0000 UTC m=+810.189693964"
Oct 30 19:32:26.209243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.209211    5033 generic.go:356] "Generic (PLEG): container finished" podID="a39c92d4-23f7-4ef5-abe1-2b083a360072" containerID="10b51867b409c47731ccebe67bcd03b4512cce20b01a1b1d6a6d610daa8ce6a2" exitCode=0
Oct 30 19:32:26.209657 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.209317    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn" event={"ID":"a39c92d4-23f7-4ef5-abe1-2b083a360072","Type":"ContainerDied","Data":"10b51867b409c47731ccebe67bcd03b4512cce20b01a1b1d6a6d610daa8ce6a2"}
Oct 30 19:32:26.415728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.415705    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:26.498411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.498342    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\" (UniqueName: \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\") pod \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") "
Oct 30 19:32:26.498573 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.498415    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\") pod \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") "
Oct 30 19:32:26.498640 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.498592    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\") pod \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\" (UID: \"f0273b89-0592-4099-ad0a-93e4c5e4b0ad\") "
Oct 30 19:32:26.499100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.499023    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca" pod "f0273b89-0592-4099-ad0a-93e4c5e4b0ad" (UID: "f0273b89-0592-4099-ad0a-93e4c5e4b0ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:26.499418 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.499394    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:26.500644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.500615    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca" pod "f0273b89-0592-4099-ad0a-93e4c5e4b0ad" (UID: "f0273b89-0592-4099-ad0a-93e4c5e4b0ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:26.500819 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.500795    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l" pod "f0273b89-0592-4099-ad0a-93e4c5e4b0ad" (UID: "f0273b89-0592-4099-ad0a-93e4c5e4b0ad"). InnerVolumeSpecName "kube-api-access-ll28l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:26.601192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.601159    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll28l\" (UniqueName: \"kubernetes.io/projected/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-kube-api-access-ll28l\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:26.601192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:26.601195    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f0273b89-0592-4099-ad0a-93e4c5e4b0ad-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.219778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.219394    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"
Oct 30 19:32:27.219778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.219447    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4" event={"ID":"f0273b89-0592-4099-ad0a-93e4c5e4b0ad","Type":"ContainerDied","Data":"7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb"}
Oct 30 19:32:27.219778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.219510    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ea0dcf4dffb9fd0467292086b58878efe8065b6a3ab324cdb1073b5296986bb"
Oct 30 19:32:27.445984 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.445915    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:27.514852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.514819    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\") pod \"a39c92d4-23f7-4ef5-abe1-2b083a360072\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") "
Oct 30 19:32:27.515031 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.514977    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\") pod \"a39c92d4-23f7-4ef5-abe1-2b083a360072\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") "
Oct 30 19:32:27.515098 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.515048    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\" (UniqueName: \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\") pod \"a39c92d4-23f7-4ef5-abe1-2b083a360072\" (UID: \"a39c92d4-23f7-4ef5-abe1-2b083a360072\") "
Oct 30 19:32:27.515305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.515277    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca" pod "a39c92d4-23f7-4ef5-abe1-2b083a360072" (UID: "a39c92d4-23f7-4ef5-abe1-2b083a360072"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.516198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.516169    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.517501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.517476    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw" pod "a39c92d4-23f7-4ef5-abe1-2b083a360072" (UID: "a39c92d4-23f7-4ef5-abe1-2b083a360072"). InnerVolumeSpecName "kube-api-access-26chw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:27.518301 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.518276    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca" pod "a39c92d4-23f7-4ef5-abe1-2b083a360072" (UID: "a39c92d4-23f7-4ef5-abe1-2b083a360072"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.617649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.617621    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a39c92d4-23f7-4ef5-abe1-2b083a360072-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.617649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:27.617649    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26chw\" (UniqueName: \"kubernetes.io/projected/a39c92d4-23f7-4ef5-abe1-2b083a360072-kube-api-access-26chw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:28.229252 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.229212    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn" event={"ID":"a39c92d4-23f7-4ef5-abe1-2b083a360072","Type":"ContainerDied","Data":"55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb"}
Oct 30 19:32:28.229603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.229281    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55544b6b24d826912a70e98bc8b470b49cb0ff3622fd8c12c0db1b58d00750bb"
Oct 30 19:32:28.229603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.229230    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"
Oct 30 19:32:28.481666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.481575    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:32:28.482620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482596    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f0273b89-0592-4099-ad0a-93e4c5e4b0ad" containerName="revoke-keys"
Oct 30 19:32:28.482749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482622    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0273b89-0592-4099-ad0a-93e4c5e4b0ad" containerName="revoke-keys"
Oct 30 19:32:28.482749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482709    5033 cpu_manager.go:414] "containerMap: removing container" podUID="a39c92d4-23f7-4ef5-abe1-2b083a360072" containerName="revoke-keys"
Oct 30 19:32:28.482749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482719    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a39c92d4-23f7-4ef5-abe1-2b083a360072" containerName="revoke-keys"
Oct 30 19:32:28.482915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482899    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="a39c92d4-23f7-4ef5-abe1-2b083a360072" containerName="revoke-keys"
Oct 30 19:32:28.482967 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.482926    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0273b89-0592-4099-ad0a-93e4c5e4b0ad" containerName="revoke-keys"
Oct 30 19:32:28.487820 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.487791    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.507580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.507555    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:32:28.531015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.530986    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531063    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531225 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531156    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531328 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531308    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531372    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531455    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531483    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-572p6\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531528    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.531637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.531610    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.532755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.532729    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:32:28.540701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.540662    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:32:28.572374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.572346    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:32:28.633381 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633351    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633395    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633449    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633474    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-572p6\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633516    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633564    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633740    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633838    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.633923 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633872    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.634026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633960    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.634026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.633987    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.634260 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.634233    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.634388 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.634286    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.634647 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.634624    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.635858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.635837    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.636126 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.636110    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.643173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.643153    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.643280 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.643258    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-572p6\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\") pod \"e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") " pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:28.813322 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:28.813243    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:29.117930 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:29.117899    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc584e0cc_8bb3_4ce2_8b2a_5beaa6c8741d.slice/crio-4f8579c97856acce0d3bdb1013f1069e43ed4708cc0b40eec92bb9dc5c3aca9b WatchSource:0}: Error finding container 4f8579c97856acce0d3bdb1013f1069e43ed4708cc0b40eec92bb9dc5c3aca9b: Status 404 returned error can't find the container with id 4f8579c97856acce0d3bdb1013f1069e43ed4708cc0b40eec92bb9dc5c3aca9b
Oct 30 19:32:29.118527 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.118498    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:32:29.121026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.120985    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:29.121108 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.121071    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:29.121149 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.121115    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:29.252700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.244187    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" event={"ID":"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d","Type":"ContainerStarted","Data":"6dba2239835baaf4f4559185ae7743e34639eb2dd9b1e3b1565b6d3fa5513552"}
Oct 30 19:32:29.252700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.244279    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" event={"ID":"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d","Type":"ContainerStarted","Data":"4f8579c97856acce0d3bdb1013f1069e43ed4708cc0b40eec92bb9dc5c3aca9b"}
Oct 30 19:32:29.813932 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.813900    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:29.838824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.838772    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podStartSLOduration=1.838757641 podStartE2EDuration="1.838757641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:29.288049812 +0000 UTC m=+814.257690648" watchObservedRunningTime="2026-10-30 19:32:29.838757641 +0000 UTC m=+814.808398476"
Oct 30 19:32:29.843644 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.843617    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"]
Oct 30 19:32:29.848251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.848236    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:29.858368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.857940    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:32:29.858368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.858006    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:29.866051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.864100    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"]
Oct 30 19:32:29.973311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.971746    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhktc\" (UniqueName: \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:29.973311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.971874    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:29.973311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:29.972032    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.078821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.078726    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhktc\" (UniqueName: \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.079592 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.079566    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.079845 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.079827    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.081895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.081240    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.083497 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.083463    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.089648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.089627    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhktc\" (UniqueName: \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\") pod \"maas-api-revoke-sub-09ef2027-sfmpb\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.176695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.176636    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:30.594118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.594065    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"]
Oct 30 19:32:30.815468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.815426    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:30.815614 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:30.815495    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:31.066321 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.066266    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" containerID="cri-o://c2ee3e12e051e383c1621549bd706a6750ae78c6a6d8ab6fec5aaf855af044b3" gracePeriod=45
Oct 30 19:32:31.346667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.337454    5033 generic.go:356] "Generic (PLEG): container finished" podID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerID="c2ee3e12e051e383c1621549bd706a6750ae78c6a6d8ab6fec5aaf855af044b3" exitCode=0
Oct 30 19:32:31.346667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.337820    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" event={"ID":"9a667eb7-4853-46d8-bacb-e27de8464c1c","Type":"ContainerDied","Data":"c2ee3e12e051e383c1621549bd706a6750ae78c6a6d8ab6fec5aaf855af044b3"}
Oct 30 19:32:31.427878 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.427756    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb" event={"ID":"83d53f99-f75b-47fa-9870-8f28595e0344","Type":"ContainerStarted","Data":"f018b5f6f6ac052471c15adbe9f31fb96e60fc0d5cdeca387424261a02bc664c"}
Oct 30 19:32:31.427878 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.427820    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb" event={"ID":"83d53f99-f75b-47fa-9870-8f28595e0344","Type":"ContainerStarted","Data":"1b39047a3d0ddc146a46786195fc71d17d5ebc2e0e8e3189f057ef466e32001c"}
Oct 30 19:32:31.501651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.501605    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb" podStartSLOduration=2.501590494 podStartE2EDuration="2.501590494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:31.488552608 +0000 UTC m=+816.458193446" watchObservedRunningTime="2026-10-30 19:32:31.501590494 +0000 UTC m=+816.471231356"
Oct 30 19:32:31.509753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.509413    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:32:31.545661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.545593    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\" (UniqueName: \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.546365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.546341    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.546790 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.546770    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.546996 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.546977    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.547220 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.547200    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.547396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.547382    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\") pod \"9a667eb7-4853-46d8-bacb-e27de8464c1c\" (UID: \"9a667eb7-4853-46d8-bacb-e27de8464c1c\") "
Oct 30 19:32:31.549549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.546875    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.549549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.547057    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.549549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.549320    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "kube-api-access-9b4fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:31.551809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.551772    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.594959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.593966    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:31.594959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.593988    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm" pod "9a667eb7-4853-46d8-bacb-e27de8464c1c" (UID: "9a667eb7-4853-46d8-bacb-e27de8464c1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.678296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.678260    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.678503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.678489    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a667eb7-4853-46d8-bacb-e27de8464c1c-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.678597 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.678584    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.678698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.678671    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.685006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.684983    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b4fg\" (UniqueName: \"kubernetes.io/projected/9a667eb7-4853-46d8-bacb-e27de8464c1c-kube-api-access-9b4fg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.685133 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.685120    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a667eb7-4853-46d8-bacb-e27de8464c1c-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.813853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.813773    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:31.813853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:31.813834    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:32.104404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.104301    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:32.435449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.435363    5033 generic.go:356] "Generic (PLEG): container finished" podID="83d53f99-f75b-47fa-9870-8f28595e0344" containerID="f018b5f6f6ac052471c15adbe9f31fb96e60fc0d5cdeca387424261a02bc664c" exitCode=0
Oct 30 19:32:32.435611 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.435445    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb" event={"ID":"83d53f99-f75b-47fa-9870-8f28595e0344","Type":"ContainerDied","Data":"f018b5f6f6ac052471c15adbe9f31fb96e60fc0d5cdeca387424261a02bc664c"}
Oct 30 19:32:32.437097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.437062    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"
Oct 30 19:32:32.437097 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.437087    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" event={"ID":"9a667eb7-4853-46d8-bacb-e27de8464c1c","Type":"ContainerDied","Data":"f4e2bd00494136c6e91569f99d57a97f2db0fca3b2f83e26105218e10bc15726"}
Oct 30 19:32:32.437300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.437143    5033 scope.go:122] "RemoveContainer" containerID="c2ee3e12e051e383c1621549bd706a6750ae78c6a6d8ab6fec5aaf855af044b3"
Oct 30 19:32:32.500626 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.500587    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:32.516016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.515985    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:32:32.525978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.525766    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-d55178/simulator-w2-d55178-kserve-79ff794f47-gbkk4"]
Oct 30 19:32:32.709832 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.709795    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"]
Oct 30 19:32:32.710836 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.710813    5033 cpu_manager.go:414] "containerMap: removing container" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main"
Oct 30 19:32:32.710931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.710838    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main"
Oct 30 19:32:32.711142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.711124    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" containerName="main"
Oct 30 19:32:32.718637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.717900    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.719140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.719121    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"]
Oct 30 19:32:32.814186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.814150    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:32.814376 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.814206    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:32.833883 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.833856    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgms\" (UniqueName: \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.834003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.833934    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.834110 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.834088    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.935395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.935359    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.935570 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.935454    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgms\" (UniqueName: \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.935570 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.935540    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.936250 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.936219    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.937996 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.937973    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:32.993787 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:32.993705    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgms\" (UniqueName: \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\") pod \"maas-api-revoke-sub-0851b971-vsmmg\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:33.035838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.035805    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:33.220764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.220734    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a667eb7-4853-46d8-bacb-e27de8464c1c" path="/var/lib/kubelet/pods/9a667eb7-4853-46d8-bacb-e27de8464c1c/volumes"
Oct 30 19:32:33.401747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.396848    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"]
Oct 30 19:32:33.410896 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:33.410804    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8533de32_5e10_4779_a285_89dd45de080e.slice/crio-bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1 WatchSource:0}: Error finding container bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1: Status 404 returned error can't find the container with id bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1
Oct 30 19:32:33.446976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.446938    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg" event={"ID":"8533de32-5e10-4779-a285-89dd45de080e","Type":"ContainerStarted","Data":"bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1"}
Oct 30 19:32:33.703543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.703514    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:33.754700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.749739    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\") pod \"83d53f99-f75b-47fa-9870-8f28595e0344\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") "
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.794265    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca" pod "83d53f99-f75b-47fa-9870-8f28595e0344" (UID: "83d53f99-f75b-47fa-9870-8f28595e0344"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.814136    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.814593    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.814897    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\") pod \"83d53f99-f75b-47fa-9870-8f28595e0344\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") "
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.815046    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\" (UniqueName: \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\") pod \"83d53f99-f75b-47fa-9870-8f28595e0344\" (UID: \"83d53f99-f75b-47fa-9870-8f28595e0344\") "
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.817123    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:33.820484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.820096    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca" pod "83d53f99-f75b-47fa-9870-8f28595e0344" (UID: "83d53f99-f75b-47fa-9870-8f28595e0344"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:33.822035 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.822015    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc" pod "83d53f99-f75b-47fa-9870-8f28595e0344" (UID: "83d53f99-f75b-47fa-9870-8f28595e0344"). InnerVolumeSpecName "kube-api-access-qhktc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:33.906828 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.906799    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:32:33.918829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.918804    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83d53f99-f75b-47fa-9870-8f28595e0344-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:33.919061 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.919048    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhktc\" (UniqueName: \"kubernetes.io/projected/83d53f99-f75b-47fa-9870-8f28595e0344-kube-api-access-qhktc\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:33.925633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:33.925604    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:32:34.517695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.517242    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb" event={"ID":"83d53f99-f75b-47fa-9870-8f28595e0344","Type":"ContainerDied","Data":"1b39047a3d0ddc146a46786195fc71d17d5ebc2e0e8e3189f057ef466e32001c"}
Oct 30 19:32:34.517695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.517325    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b39047a3d0ddc146a46786195fc71d17d5ebc2e0e8e3189f057ef466e32001c"
Oct 30 19:32:34.517695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.517447    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"
Oct 30 19:32:34.520188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.520163    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg" event={"ID":"8533de32-5e10-4779-a285-89dd45de080e","Type":"ContainerStarted","Data":"cf3f76bca4a37c302ded3c2632514baf42a6a63b3349f7df1d9b51d75a3eeb01"}
Oct 30 19:32:34.540588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.540538    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg" podStartSLOduration=2.540522174 podStartE2EDuration="2.540522174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:34.539279949 +0000 UTC m=+819.508920785" watchObservedRunningTime="2026-10-30 19:32:34.540522174 +0000 UTC m=+819.510163011"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.804260    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"]
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.806345    5033 cpu_manager.go:414] "containerMap: removing container" podUID="83d53f99-f75b-47fa-9870-8f28595e0344" containerName="revoke-keys"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.806365    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d53f99-f75b-47fa-9870-8f28595e0344" containerName="revoke-keys"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.806577    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d53f99-f75b-47fa-9870-8f28595e0344" containerName="revoke-keys"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.813558    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.813599    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.815347    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.816367    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"]
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.823927    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.837076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.832352    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"]
Oct 30 19:32:34.838460 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.838404    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"]
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846107    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846220    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846269    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846415    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4bxm\" (UniqueName: \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846443    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.846713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.846498    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx29r\" (UniqueName: \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.947877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.947838    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.949045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.949020    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4bxm\" (UniqueName: \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.949241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.948830    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.949323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.949225    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.949399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.949315    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx29r\" (UniqueName: \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.949399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.949357    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:34.949499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.949487    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:34.967638 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:34.967610    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:35.028555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.028484    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4bxm\" (UniqueName: \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:35.030797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.030736    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46a76b65-kw55x\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:35.033657 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.033611    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx29r\" (UniqueName: \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:35.037550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.037496    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a1c28c5-nx5dl\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:35.204760 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:35.185843    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:32:35.233717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.219727    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:35.233717 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.228593    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:35.622804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.619263    5033 generic.go:356] "Generic (PLEG): container finished" podID="8533de32-5e10-4779-a285-89dd45de080e" containerID="cf3f76bca4a37c302ded3c2632514baf42a6a63b3349f7df1d9b51d75a3eeb01" exitCode=0
Oct 30 19:32:35.622804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.619650    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg" event={"ID":"8533de32-5e10-4779-a285-89dd45de080e","Type":"ContainerDied","Data":"cf3f76bca4a37c302ded3c2632514baf42a6a63b3349f7df1d9b51d75a3eeb01"}
Oct 30 19:32:35.819478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.819385    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:32:35.819478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.819446    5033 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 19:32:35.866473 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.866440    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"]
Oct 30 19:32:35.868281 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:35.868237    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709ff886_5f5a_48e9_8940_4dfd197c9a6f.slice/crio-08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416 WatchSource:0}: Error finding container 08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416: Status 404 returned error can't find the container with id 08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416
Oct 30 19:32:35.927339 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:35.927301    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ce3811_25c2_4220_83a3_374f1c84248b.slice/crio-051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e WatchSource:0}: Error finding container 051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e: Status 404 returned error can't find the container with id 051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e
Oct 30 19:32:35.928117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:35.928093    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"]
Oct 30 19:32:36.635311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.635272    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl" event={"ID":"709ff886-5f5a-48e9-8940-4dfd197c9a6f","Type":"ContainerStarted","Data":"aaaac34d5db2b6a58a289ed853eda1603afecde4d9afb6b8bebc87359fc24571"}
Oct 30 19:32:36.635566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.635529    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl" event={"ID":"709ff886-5f5a-48e9-8940-4dfd197c9a6f","Type":"ContainerStarted","Data":"08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416"}
Oct 30 19:32:36.637470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.637440    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x" event={"ID":"78ce3811-25c2-4220-83a3-374f1c84248b","Type":"ContainerStarted","Data":"b1466db9a7d101a94aef43c3d5056083e631b3976f305f381bc47072c066565d"}
Oct 30 19:32:36.637572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.637493    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x" event={"ID":"78ce3811-25c2-4220-83a3-374f1c84248b","Type":"ContainerStarted","Data":"051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e"}
Oct 30 19:32:36.662760 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.662703    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl" podStartSLOduration=2.662668325 podStartE2EDuration="2.662668325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:36.658383348 +0000 UTC m=+821.628024185" watchObservedRunningTime="2026-10-30 19:32:36.662668325 +0000 UTC m=+821.632309161"
Oct 30 19:32:36.681547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.681479    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x" podStartSLOduration=2.681446126 podStartE2EDuration="2.681446126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:36.680714689 +0000 UTC m=+821.650355527" watchObservedRunningTime="2026-10-30 19:32:36.681446126 +0000 UTC m=+821.651086962"
Oct 30 19:32:36.901897 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:36.901865    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:37.015223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.015139    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\") pod \"8533de32-5e10-4779-a285-89dd45de080e\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") "
Oct 30 19:32:37.015354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.015336    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\" (UniqueName: \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\") pod \"8533de32-5e10-4779-a285-89dd45de080e\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") "
Oct 30 19:32:37.015419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.015407    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\") pod \"8533de32-5e10-4779-a285-89dd45de080e\" (UID: \"8533de32-5e10-4779-a285-89dd45de080e\") "
Oct 30 19:32:37.015902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.015866    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca" pod "8533de32-5e10-4779-a285-89dd45de080e" (UID: "8533de32-5e10-4779-a285-89dd45de080e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:37.017431 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.017406    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca" pod "8533de32-5e10-4779-a285-89dd45de080e" (UID: "8533de32-5e10-4779-a285-89dd45de080e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:37.017628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.017612    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms" pod "8533de32-5e10-4779-a285-89dd45de080e" (UID: "8533de32-5e10-4779-a285-89dd45de080e"). InnerVolumeSpecName "kube-api-access-rfgms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:37.117701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.117640    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfgms\" (UniqueName: \"kubernetes.io/projected/8533de32-5e10-4779-a285-89dd45de080e-kube-api-access-rfgms\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:37.118048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.117992    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8533de32-5e10-4779-a285-89dd45de080e-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:37.118048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.118027    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8533de32-5e10-4779-a285-89dd45de080e-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:37.239071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.239029    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:37.239484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.239458    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:37.240330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.240305    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:32:37.271255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.271163    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:32:37.301358 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.301325    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:32:37.648656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.648581    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"
Oct 30 19:32:37.648656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.648607    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg" event={"ID":"8533de32-5e10-4779-a285-89dd45de080e","Type":"ContainerDied","Data":"bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1"}
Oct 30 19:32:37.648656 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.648659    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf669b755a2614e6cb7495bff32fba64c4d6d211b649f79fce9a9217e25966e1"
Oct 30 19:32:37.650530 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.650504    5033 generic.go:356] "Generic (PLEG): container finished" podID="709ff886-5f5a-48e9-8940-4dfd197c9a6f" containerID="aaaac34d5db2b6a58a289ed853eda1603afecde4d9afb6b8bebc87359fc24571" exitCode=0
Oct 30 19:32:37.650808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.650576    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl" event={"ID":"709ff886-5f5a-48e9-8940-4dfd197c9a6f","Type":"ContainerDied","Data":"aaaac34d5db2b6a58a289ed853eda1603afecde4d9afb6b8bebc87359fc24571"}
Oct 30 19:32:37.653315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.653288    5033 generic.go:356] "Generic (PLEG): container finished" podID="78ce3811-25c2-4220-83a3-374f1c84248b" containerID="b1466db9a7d101a94aef43c3d5056083e631b3976f305f381bc47072c066565d" exitCode=0
Oct 30 19:32:37.653444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:37.653345    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x" event={"ID":"78ce3811-25c2-4220-83a3-374f1c84248b","Type":"ContainerDied","Data":"b1466db9a7d101a94aef43c3d5056083e631b3976f305f381bc47072c066565d"}
Oct 30 19:32:38.968753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:38.943863    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing" containerID="cri-o://60b83c5026f0cc9bf0504a8fa2ca694e7cf3b48db88f3f7fc3e57eec19efec2c" gracePeriod=25
Oct 30 19:32:38.968753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:38.945346    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing" containerID="cri-o://47160548286b7a060863053e28d8386e9cd5cd228140c044c6aac3be6f00bf9f" gracePeriod=25
Oct 30 19:32:39.646701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.644779    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:39.800708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.799802    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:39.808822 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.805752    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:39.862079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.847754    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:39.972712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.969098    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:39.978210 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:39.978173    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.245818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.245768    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.347945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.346822    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:32:40.379744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.379563    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.658700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.656560    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:32:40.702916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.692466    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:32:40.702916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.693308    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:32:40.851749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.847362    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.851749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.847501    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:40.867822 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.852119    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.867822 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.852713    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:32:40.939818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.852065    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:40.965217 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:40.965186    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:32:41.044774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.027473    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.053299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.053275    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:32:41.256710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.250985    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.256710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.254590    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:41.353719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.348550    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.353719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.350282    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:32:41.429577 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.413194    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.429577 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.428357    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:32:41.446796 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.446766    5033 generic.go:356] "Generic (PLEG): container finished" podID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerID="eeb5398f42908c8f2403175aa7f6ef1bc683aa3215ca294c0ba2853cd0a405d6" exitCode=0
Oct 30 19:32:41.467934 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.458326    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" event={"ID":"ce29ab48-fb2c-47eb-b024-9e0744f9c869","Type":"ContainerDied","Data":"eeb5398f42908c8f2403175aa7f6ef1bc683aa3215ca294c0ba2853cd0a405d6"}
Oct 30 19:32:41.490637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.490181    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:32:41.490637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.490250    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.675849 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.675807    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:32:41.752772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.733837    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:32:41.803708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.773010    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:32:41.898045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.857761    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:41.898045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.881762    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:32:41.949609 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:41.948469    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.069784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.044363    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.085344 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.049982    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.176083 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.176039    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.237120 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.237075    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:42.237573 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.237544    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:42.271988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.267559    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/health\": context deadline exceeded"
Oct 30 19:32:42.306004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.305963    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:32:42.425369 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.412182    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.428104 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.428058    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.581890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.571980    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:42.720707 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.720506    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.722319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.722283    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:42.743770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.741203    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829224    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\") pod \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") "
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829379    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\") pod \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") "
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829428    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\" (UniqueName: \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\") pod \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\" (UID: \"709ff886-5f5a-48e9-8940-4dfd197c9a6f\") "
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829509    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\") pod \"78ce3811-25c2-4220-83a3-374f1c84248b\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") "
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829556    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\" (UniqueName: \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\") pod \"78ce3811-25c2-4220-83a3-374f1c84248b\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") "
Oct 30 19:32:42.831719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.829782    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\") pod \"78ce3811-25c2-4220-83a3-374f1c84248b\" (UID: \"78ce3811-25c2-4220-83a3-374f1c84248b\") "
Oct 30 19:32:42.864705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.860939    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:32:42.920716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.915916    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:32:42.920716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.915995    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:42.977351 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.977253    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.353034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.978557    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca" pod "78ce3811-25c2-4220-83a3-374f1c84248b" (UID: "78ce3811-25c2-4220-83a3-374f1c84248b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:43.353034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.978892    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca" pod "709ff886-5f5a-48e9-8940-4dfd197c9a6f" (UID: "709ff886-5f5a-48e9-8940-4dfd197c9a6f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:43.353034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.337340    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca" pod "78ce3811-25c2-4220-83a3-374f1c84248b" (UID: "78ce3811-25c2-4220-83a3-374f1c84248b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:43.353034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.338334    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r" pod "709ff886-5f5a-48e9-8940-4dfd197c9a6f" (UID: "709ff886-5f5a-48e9-8940-4dfd197c9a6f"). InnerVolumeSpecName "kube-api-access-lx29r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:43.353034 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.316003    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.373330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:42.978958    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:32:43.387724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.381847    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm" pod "78ce3811-25c2-4220-83a3-374f1c84248b" (UID: "78ce3811-25c2-4220-83a3-374f1c84248b"). InnerVolumeSpecName "kube-api-access-g4bxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:43.401054 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.073349    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.402543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.108819    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.403202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.403179    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:32:43.434990 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.434951    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca" pod "709ff886-5f5a-48e9-8940-4dfd197c9a6f" (UID: "709ff886-5f5a-48e9-8940-4dfd197c9a6f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:43.672598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.672485    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:43.718850 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.676898    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.720231 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.690990    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.61:50052\" within 1s: context deadline exceeded"
Oct 30 19:32:43.720472 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:43.691085    5033 logging.go:55] [core] [Channel #368 SubChannel #369]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.61:50052", ServerName: "10.132.0.61:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.61:50052: operation was canceled"
Oct 30 19:32:43.761703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.748175    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:43.761703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.748516    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766757    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766790    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766806    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/709ff886-5f5a-48e9-8940-4dfd197c9a6f-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766820    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx29r\" (UniqueName: \"kubernetes.io/projected/709ff886-5f5a-48e9-8940-4dfd197c9a6f-kube-api-access-lx29r\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766835    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/78ce3811-25c2-4220-83a3-374f1c84248b-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.788699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.766848    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4bxm\" (UniqueName: \"kubernetes.io/projected/78ce3811-25c2-4220-83a3-374f1c84248b-kube-api-access-g4bxm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:43.829121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.806960    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" containerID="cri-o://a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749" gracePeriod=45
Oct 30 19:32:43.829121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.807123    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" containerID="cri-o://230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980" gracePeriod=45
Oct 30 19:32:43.880593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.880493    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.906156    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.909495    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:43.995808    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.002382    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x" event={"ID":"78ce3811-25c2-4220-83a3-374f1c84248b","Type":"ContainerDied","Data":"051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e"}
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.002441    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="051178c31d21cead871bd245198c1b27b3fb7d30eb234aa60d99c1ed2fdbda2e"
Oct 30 19:32:44.006006 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.002473    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:32:44.096702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.095124    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" containerID="cri-o://a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347" gracePeriod=45
Oct 30 19:32:44.244184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.169018    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl" event={"ID":"709ff886-5f5a-48e9-8940-4dfd197c9a6f","Type":"ContainerDied","Data":"08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416"}
Oct 30 19:32:44.244184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.169080    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08b4bf1ec8b72fd7897723ce0eae1514d47c31794bbbeea992e951039b278416"
Oct 30 19:32:44.244184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.169341    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"
Oct 30 19:32:44.244184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.188410    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\") pod \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") "
Oct 30 19:32:44.244184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.188586    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\" (UniqueName: \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\") pod \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\" (UID: \"ce29ab48-fb2c-47eb-b024-9e0744f9c869\") "
Oct 30 19:32:44.355296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.286986    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume" pod "ce29ab48-fb2c-47eb-b024-9e0744f9c869" (UID: "ce29ab48-fb2c-47eb-b024-9e0744f9c869"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:44.355296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.315799    5033 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce29ab48-fb2c-47eb-b024-9e0744f9c869-plugins-config-volume\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:44.373699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.369479    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 19:32:44.373699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.369853    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:44.400347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.398268    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 19:32:44.418278 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.418238    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:44.418869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.418840    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:32:44.523714 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.504399    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": read tcp 10.132.0.2:59820->10.132.0.88:8000: read: connection reset by peer"
Oct 30 19:32:44.711039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.701141    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": dial tcp 10.132.0.88:8000: connect: connection refused"
Oct 30 19:32:44.717086 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.713917    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": read tcp 10.132.0.2:41364->10.132.0.64:8000: read: connection reset by peer"
Oct 30 19:32:44.717280 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.715389    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:44.732474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.721834    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7" pod "ce29ab48-fb2c-47eb-b024-9e0744f9c869" (UID: "ce29ab48-fb2c-47eb-b024-9e0744f9c869"). InnerVolumeSpecName "kube-api-access-sdrf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:44.732474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.721919    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:44.733254 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.733227    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:32:44.776227 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.751956    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdrf7\" (UniqueName: \"kubernetes.io/projected/ce29ab48-fb2c-47eb-b024-9e0744f9c869-kube-api-access-sdrf7\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:44.869698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:44.848336    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": dial tcp 10.132.0.64:8000: connect: connection refused"
Oct 30 19:32:45.388699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.383793    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:45.388699 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.384078    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:45.441741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.427373    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:32:45.500068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.461012    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:45.542702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.461716    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:32:45.543339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.486978    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:45.748699 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:45.714666    5033 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ce3811_25c2_4220_83a3_374f1c84248b.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:32:45.836901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.736121    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:45.846614 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.736160    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" probeResult="failure" output="Get \"https://10.132.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:32:45.850867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.736289    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:32:45.877884 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.780029    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": dial tcp 10.132.0.88:8000: connect: connection refused"
Oct 30 19:32:45.877884 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.800519    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"
Oct 30 19:32:45.878204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.817579    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994" event={"ID":"ce29ab48-fb2c-47eb-b024-9e0744f9c869","Type":"ContainerDied","Data":"0aa871957f1f3a933b78109345cddbb14f2cc56e8e43ff85aef6e07226d55006"}
Oct 30 19:32:45.878389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.878373    5033 scope.go:122] "RemoveContainer" containerID="eeb5398f42908c8f2403175aa7f6ef1bc683aa3215ca294c0ba2853cd0a405d6"
Oct 30 19:32:45.878841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.817654    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": dial tcp 10.132.0.64:8000: connect: connection refused"
Oct 30 19:32:45.914360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:45.914325    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:32:46.068894 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.068804    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:46.287277 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.281840    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:32:46.287277 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.282291    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy" containerID="cri-o://b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00" gracePeriod=30
Oct 30 19:32:46.322227 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.321133    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:32:46.322227 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.321576    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy" containerID="cri-o://9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b" gracePeriod=30
Oct 30 19:32:46.430723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.421956    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:46.430723 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.426932    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:32:46.559821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.555780    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": context deadline exceeded"
Oct 30 19:32:46.559821 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.556562    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": dial tcp 10.132.0.97:8000: connect: connection refused"
Oct 30 19:32:46.600750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.594648    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": dial tcp 10.132.0.88:8000: connect: connection refused"
Oct 30 19:32:46.625754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.623038    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:46.834519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.823437    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": dial tcp 10.132.0.64:8000: connect: connection refused"
Oct 30 19:32:46.834519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.827048    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:32:46.834519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.831850    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:32:46.862394 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.862304    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-9f2d57-56656bc6c5-25994"]
Oct 30 19:32:46.862669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.862648    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:32:46.899850 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.876798    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:46.979700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.975981    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main" probeResult="failure" output="Get \"https://10.132.0.97:8000/ready\": dial tcp 10.132.0.97:8000: connect: connection refused"
Oct 30 19:32:46.990304 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:46.989773    5033 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:47.230490 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.230454    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" path="/var/lib/kubelet/pods/ce29ab48-fb2c-47eb-b024-9e0744f9c869/volumes"
Oct 30 19:32:47.240786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.240756    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:32:47.553703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.550594    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:32:47.561698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.559291    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654079    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654137    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654176    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\" (UniqueName: \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654228    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\" (UniqueName: \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654293    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654339    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654379    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654572    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654659    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\" (UniqueName: \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654734    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654835    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\") pod \"e01494b8-4ee0-4097-88ef-7944f98cfc39\" (UID: \"e01494b8-4ee0-4097-88ef-7944f98cfc39\") "
Oct 30 19:32:47.656275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.654889    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\") pod \"a624dad3-3364-4c6c-a18c-382928c0e89c\" (UID: \"a624dad3-3364-4c6c-a18c-382928c0e89c\") "
Oct 30 19:32:47.657014 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.656416    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.657014 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.656969    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.657167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.657145    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.657330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.657311    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.657830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.657804    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.658283 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.658258    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.660831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.660806    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "kube-api-access-l65d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:47.680933 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.680905    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.681089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.681069    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:47.682081 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.682055    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "kube-api-access-6lc26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:47.682441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.682395    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs" pod "a624dad3-3364-4c6c-a18c-382928c0e89c" (UID: "a624dad3-3364-4c6c-a18c-382928c0e89c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:47.690805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.690777    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm" pod "e01494b8-4ee0-4097-88ef-7944f98cfc39" (UID: "e01494b8-4ee0-4097-88ef-7944f98cfc39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.782496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.782457    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785035    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l65d9\" (UniqueName: \"kubernetes.io/projected/a624dad3-3364-4c6c-a18c-382928c0e89c-kube-api-access-l65d9\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785217    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785330    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785431 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785422    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a624dad3-3364-4c6c-a18c-382928c0e89c-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785492    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785578 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785568    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785635    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lc26\" (UniqueName: \"kubernetes.io/projected/e01494b8-4ee0-4097-88ef-7944f98cfc39-kube-api-access-6lc26\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785731    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e01494b8-4ee0-4097-88ef-7944f98cfc39-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785812    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785882    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a624dad3-3364-4c6c-a18c-382928c0e89c-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.785963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.785954    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e01494b8-4ee0-4097-88ef-7944f98cfc39-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.830228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.830203    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887030    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\" (UniqueName: \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887099    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887195    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\" (UniqueName: \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887319    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887339    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.887945 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.887395    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\") pod \"77d09205-17a9-4dfc-98d1-6440f6991154\" (UID: \"77d09205-17a9-4dfc-98d1-6440f6991154\") "
Oct 30 19:32:47.888354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.888327    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.888578 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.888557    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.889011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.888978    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.891075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.891049    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:47.891311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.891292    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "kube-api-access-mvwpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:47.892808 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.892785    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm" pod "77d09205-17a9-4dfc-98d1-6440f6991154" (UID: "77d09205-17a9-4dfc-98d1-6440f6991154"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:47.912904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.912836    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:32:47.992556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992519    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.992556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992557    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.992789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992570    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.992789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992582    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77d09205-17a9-4dfc-98d1-6440f6991154-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.992789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992594    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77d09205-17a9-4dfc-98d1-6440f6991154-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.992789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.992609    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvwpk\" (UniqueName: \"kubernetes.io/projected/77d09205-17a9-4dfc-98d1-6440f6991154-kube-api-access-mvwpk\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:47.995269 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.994668    5033 generic.go:356] "Generic (PLEG): container finished" podID="77d09205-17a9-4dfc-98d1-6440f6991154" containerID="a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749" exitCode=0
Oct 30 19:32:47.995842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.995443    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" event={"ID":"77d09205-17a9-4dfc-98d1-6440f6991154","Type":"ContainerDied","Data":"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"}
Oct 30 19:32:47.995842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.995487    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" event={"ID":"77d09205-17a9-4dfc-98d1-6440f6991154","Type":"ContainerDied","Data":"79918fd22e3374a118478bedd7d3f32c4d6391f6c1bbd24e99918b58c9719b27"}
Oct 30 19:32:47.995842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.995509    5033 scope.go:122] "RemoveContainer" containerID="a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"
Oct 30 19:32:47.995842 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:47.995647    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"
Oct 30 19:32:48.006275 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.006244    5033 generic.go:356] "Generic (PLEG): container finished" podID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerID="230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980" exitCode=0
Oct 30 19:32:48.006416 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.006397    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" event={"ID":"a624dad3-3364-4c6c-a18c-382928c0e89c","Type":"ContainerDied","Data":"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"}
Oct 30 19:32:48.006484 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.006439    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln" event={"ID":"a624dad3-3364-4c6c-a18c-382928c0e89c","Type":"ContainerDied","Data":"d4769b3a42a948588fd4093260a139b46d14f2dc7898c706e59b7c45ed341b7d"}
Oct 30 19:32:48.006558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.006508    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"
Oct 30 19:32:48.012696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.011949    5033 generic.go:356] "Generic (PLEG): container finished" podID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerID="a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347" exitCode=0
Oct 30 19:32:48.012696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.012025    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" event={"ID":"e01494b8-4ee0-4097-88ef-7944f98cfc39","Type":"ContainerDied","Data":"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"}
Oct 30 19:32:48.012696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.012051    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg" event={"ID":"e01494b8-4ee0-4097-88ef-7944f98cfc39","Type":"ContainerDied","Data":"b7c75e3a481f67da2abb96b3a0a03157b200ae36cb6984855c3712e9007d1593"}
Oct 30 19:32:48.012696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.012104    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"
Oct 30 19:32:48.039565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.039541    5033 scope.go:122] "RemoveContainer" containerID="a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"
Oct 30 19:32:48.040480 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:48.040447    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749\": container with ID starting with a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749 not found: ID does not exist" containerID="a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"
Oct 30 19:32:48.040584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.040491    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749"} err="failed to get container status \"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749\": rpc error: code = NotFound desc = could not find container \"a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749\": container with ID starting with a328e79e0aacc99d631f90f9c346fc9eeffe19329c66aa5bdcadad1ae0090749 not found: ID does not exist"
Oct 30 19:32:48.040584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.040514    5033 scope.go:122] "RemoveContainer" containerID="230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"
Oct 30 19:32:48.045004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.044897    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:32:48.050950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.050926    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b"]
Oct 30 19:32:48.106793 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.106763    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:32:48.109740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.109711    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-w0-171808-kserve-695b79d8b4-wz9ln"]
Oct 30 19:32:48.114035 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.114016    5033 scope.go:122] "RemoveContainer" containerID="230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"
Oct 30 19:32:48.114411 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:48.114383    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980\": container with ID starting with 230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980 not found: ID does not exist" containerID="230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"
Oct 30 19:32:48.114517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.114498    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980"} err="failed to get container status \"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980\": rpc error: code = NotFound desc = could not find container \"230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980\": container with ID starting with 230bc5bba0d6341bebb719d7a75efd10d571421bd572d62c25256969f5248980 not found: ID does not exist"
Oct 30 19:32:48.114589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.114581    5033 scope.go:122] "RemoveContainer" containerID="a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"
Oct 30 19:32:48.126221 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.126205    5033 scope.go:122] "RemoveContainer" containerID="a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"
Oct 30 19:32:48.126485 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:48.126465    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347\": container with ID starting with a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347 not found: ID does not exist" containerID="a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"
Oct 30 19:32:48.126574 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.126491    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347"} err="failed to get container status \"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347\": rpc error: code = NotFound desc = could not find container \"a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347\": container with ID starting with a1664356b4a24fbb11116599ec725d315e1ea004e553c809cecba578fc695347 not found: ID does not exist"
Oct 30 19:32:48.131007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.130983    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:32:48.135574 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.135550    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-171808/distinct-2-w0-171808-kserve-5db94578b7-tp2qg"]
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.387017    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"]
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388044    5033 cpu_manager.go:414] "containerMap: removing container" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388067    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388128    5033 cpu_manager.go:414] "containerMap: removing container" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388136    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388147    5033 cpu_manager.go:414] "containerMap: removing container" podUID="8533de32-5e10-4779-a285-89dd45de080e" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388155    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8533de32-5e10-4779-a285-89dd45de080e" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388187    5033 cpu_manager.go:414] "containerMap: removing container" podUID="78ce3811-25c2-4220-83a3-374f1c84248b" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388210    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ce3811-25c2-4220-83a3-374f1c84248b" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388256    5033 cpu_manager.go:414] "containerMap: removing container" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388263    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388284    5033 cpu_manager.go:414] "containerMap: removing container" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388291    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388305    5033 cpu_manager.go:414] "containerMap: removing container" podUID="709ff886-5f5a-48e9-8940-4dfd197c9a6f" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388313    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="709ff886-5f5a-48e9-8940-4dfd197c9a6f" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388523    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce29ab48-fb2c-47eb-b024-9e0744f9c869" containerName="payload-processing"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388539    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="8533de32-5e10-4779-a285-89dd45de080e" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388556    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ce3811-25c2-4220-83a3-374f1c84248b" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388573    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388583    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388591    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="709ff886-5f5a-48e9-8940-4dfd197c9a6f" containerName="revoke-keys"
Oct 30 19:32:48.388903 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.388604    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" containerName="main"
Oct 30 19:32:48.404124 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.404096    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"]
Oct 30 19:32:48.404470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.404450    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.414948 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.414921    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:48.415082 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.415033    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:32:48.517856 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.517813    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.518030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.517879    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8sj\" (UniqueName: \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.518071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.518038    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.619205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.619166    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.619628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.619222    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8sj\" (UniqueName: \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.619628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.619267    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.619876 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.619853    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.621892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.621859    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.632752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.632722    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8sj\" (UniqueName: \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\") pod \"maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.741133 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.741104    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:48.814703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:48.813751    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-171808/simulator-w0-171808-kserve-669b5844f6-lrs4b" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:32:49.181525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.181446    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77d09205-17a9-4dfc-98d1-6440f6991154" path="/var/lib/kubelet/pods/77d09205-17a9-4dfc-98d1-6440f6991154/volumes"
Oct 30 19:32:49.182518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.182493    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a624dad3-3364-4c6c-a18c-382928c0e89c" path="/var/lib/kubelet/pods/a624dad3-3364-4c6c-a18c-382928c0e89c/volumes"
Oct 30 19:32:49.195017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.194993    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e01494b8-4ee0-4097-88ef-7944f98cfc39" path="/var/lib/kubelet/pods/e01494b8-4ee0-4097-88ef-7944f98cfc39/volumes"
Oct 30 19:32:49.425809 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.403538    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"]
Oct 30 19:32:49.608312 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.608261    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:32:49.608846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.608786    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" containerID="cri-o://cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8" gracePeriod=30
Oct 30 19:32:49.665147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.645525    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:32:49.665147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.645786    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" containerID="cri-o://f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962" gracePeriod=30
Oct 30 19:32:49.758867 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.758829    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"]
Oct 30 19:32:49.766147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:49.766118    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-171808-t9vbn"]
Oct 30 19:32:50.144155 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.144114    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj" event={"ID":"16155a9f-35a5-4736-ac03-95a3a8743068","Type":"ContainerStarted","Data":"357de5548de0225711bd23b2c87df4637ce85d6d18af4497302899b6eedeceb3"}
Oct 30 19:32:50.918360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.918323    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"]
Oct 30 19:32:50.923911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.923887    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:50.933398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.933376    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"]
Oct 30 19:32:50.980983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.980947    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:32:50.988578 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:50.988548    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:32:51.015061 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.014967    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7jbg\" (UniqueName: \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.015576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.015515    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.015852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.015816    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.117784 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.117749    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7jbg\" (UniqueName: \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.117983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.117848    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.117983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.117886    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.118568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.118545    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.120356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.120330    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.127340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.127315    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7jbg\" (UniqueName: \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\") pod \"maas-api-revoke-sub-4d9eef59-gdzb7\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.178106 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.178069    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a39c92d4-23f7-4ef5-abe1-2b083a360072" path="/var/lib/kubelet/pods/a39c92d4-23f7-4ef5-abe1-2b083a360072/volumes"
Oct 30 19:32:51.179242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.179203    5033 generic.go:356] "Generic (PLEG): container finished" podID="16155a9f-35a5-4736-ac03-95a3a8743068" containerID="a1b9efbbc15392d0c3e5033bc556b4e9886aebfc6d37317104504bd80d468a0c" exitCode=0
Oct 30 19:32:51.179382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.179356    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj" event={"ID":"16155a9f-35a5-4736-ac03-95a3a8743068","Type":"ContainerDied","Data":"a1b9efbbc15392d0c3e5033bc556b4e9886aebfc6d37317104504bd80d468a0c"}
Oct 30 19:32:51.265420 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.263333    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:51.569130 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:32:51.566823    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a607509_8147_453f_a594_09a247f6ced5.slice/crio-84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694 WatchSource:0}: Error finding container 84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694: Status 404 returned error can't find the container with id 84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694
Oct 30 19:32:51.574954 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.574925    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"]
Oct 30 19:32:51.926641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.926525    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:32:51.926641 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:51.926610    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:32:52.018299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.018271    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:32:52.104446 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.104365    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:52.104781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.104741    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:32:52.133094 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133067    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133233 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133182    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133260    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133311    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133359    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133479 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133457    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133555    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133603 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133598    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133650    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\") pod \"4d0be903-95d1-475a-9c0c-d32b252c81a0\" (UID: \"4d0be903-95d1-475a-9c0c-d32b252c81a0\") "
Oct 30 19:32:52.133802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133736    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.133802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.133786    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:52.134162 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134066    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.134162 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134151    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.134341 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134323    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.134405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134343    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0be903-95d1-475a-9c0c-d32b252c81a0-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.134405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134359    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.134405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134370    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.134565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.134423    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.136344 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.136321    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:52.136986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.136962    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "kube-api-access-xfw5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.137709 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.137660    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.138119 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.138105    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy" pod "4d0be903-95d1-475a-9c0c-d32b252c81a0" (UID: "4d0be903-95d1-475a-9c0c-d32b252c81a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.168229 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.168198    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:32:52.191649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.191615    5033 generic.go:356] "Generic (PLEG): container finished" podID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerID="b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00" exitCode=0
Oct 30 19:32:52.191855 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.191780    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"
Oct 30 19:32:52.192465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.192440    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerDied","Data":"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"}
Oct 30 19:32:52.192554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.192490    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds" event={"ID":"4d0be903-95d1-475a-9c0c-d32b252c81a0","Type":"ContainerDied","Data":"fed751f5a26f0a2ebfe4529ca8c7dafe67485174e1fbf0f2f8d24094a00034d4"}
Oct 30 19:32:52.192554 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.192511    5033 scope.go:122] "RemoveContainer" containerID="b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"
Oct 30 19:32:52.196335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.196196    5033 generic.go:356] "Generic (PLEG): container finished" podID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerID="9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b" exitCode=0
Oct 30 19:32:52.196335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.196297    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"
Oct 30 19:32:52.197608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.197509    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" event={"ID":"ca995be6-88c2-474e-ab02-7b6fa35f4c9c","Type":"ContainerDied","Data":"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"}
Oct 30 19:32:52.197608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.197570    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz" event={"ID":"ca995be6-88c2-474e-ab02-7b6fa35f4c9c","Type":"ContainerDied","Data":"b1183f30ba2eac71a2e90eb8286df7f0eb2f41a782bc0551c5e3d866bbd28897"}
Oct 30 19:32:52.211465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.211435    5033 scope.go:122] "RemoveContainer" containerID="cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"
Oct 30 19:32:52.211632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.211607    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7" event={"ID":"0a607509-8147-453f-a594-09a247f6ced5","Type":"ContainerStarted","Data":"90bb24e3d861ceb6032c4fad546c74b796bbef34537191f96e5fa4acb8980db1"}
Oct 30 19:32:52.211811 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.211789    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7" event={"ID":"0a607509-8147-453f-a594-09a247f6ced5","Type":"ContainerStarted","Data":"84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694"}
Oct 30 19:32:52.230639 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.230518    5033 scope.go:122] "RemoveContainer" containerID="b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"
Oct 30 19:32:52.231550 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:52.231442    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00\": container with ID starting with b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00 not found: ID does not exist" containerID="b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"
Oct 30 19:32:52.231550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.231471    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00"} err="failed to get container status \"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00\": rpc error: code = NotFound desc = could not find container \"b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00\": container with ID starting with b4561dcdd85afdec49741437d8ee2d471f608133266b8626afd6d5ec39679f00 not found: ID does not exist"
Oct 30 19:32:52.231550 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.231491    5033 scope.go:122] "RemoveContainer" containerID="cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235460    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235539    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235619    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235647    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235667    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235728    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235792    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235828    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.235915    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\") pod \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\" (UID: \"ca995be6-88c2-474e-ab02-7b6fa35f4c9c\") "
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.236693    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.236717    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfw5x\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-kube-api-access-xfw5x\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.236737    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0be903-95d1-475a-9c0c-d32b252c81a0-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.236752    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.237910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.236767    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0be903-95d1-475a-9c0c-d32b252c81a0-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.238880 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.238756    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.240463 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.239663    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.242455 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:52.242012    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06\": container with ID starting with cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06 not found: ID does not exist" containerID="cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"
Oct 30 19:32:52.242455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.242048    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06"} err="failed to get container status \"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06\": rpc error: code = NotFound desc = could not find container \"cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06\": container with ID starting with cf90bace80f1e107d2824bf2d8b97dce52ce73796ce82fe019a208bd9b2bbc06 not found: ID does not exist"
Oct 30 19:32:52.242455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.242073    5033 scope.go:122] "RemoveContainer" containerID="9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"
Oct 30 19:32:52.243162 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.242975    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.243718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.243396    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:52.249380 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.249355    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "kube-api-access-jpt8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.279065 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.244304    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.283110 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.282951    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:32:52.289285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.289258    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:52.289543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.289520    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.291071 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.291046    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token" pod "ca995be6-88c2-474e-ab02-7b6fa35f4c9c" (UID: "ca995be6-88c2-474e-ab02-7b6fa35f4c9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.295426 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.295402    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kshds"]
Oct 30 19:32:52.296174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.296159    5033 scope.go:122] "RemoveContainer" containerID="9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"
Oct 30 19:32:52.296631 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:52.296608    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b\": container with ID starting with 9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b not found: ID does not exist" containerID="9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"
Oct 30 19:32:52.296763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.296741    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b"} err="failed to get container status \"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b\": rpc error: code = NotFound desc = could not find container \"9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b\": container with ID starting with 9edbb85ae0e8b64c8ed503175dc24421c71fd68b6950de3899e5ad51e6838b7b not found: ID does not exist"
Oct 30 19:32:52.305659 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.303528    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7" podStartSLOduration=2.303515357 podStartE2EDuration="2.303515357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:52.268993484 +0000 UTC m=+837.238634321" watchObservedRunningTime="2026-10-30 19:32:52.303515357 +0000 UTC m=+837.273156192"
Oct 30 19:32:52.306383 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.306339    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main" containerID="cri-o://2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f" gracePeriod=45
Oct 30 19:32:52.335502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.335468    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main" containerID="cri-o://1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e" gracePeriod=45
Oct 30 19:32:52.338410 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338383    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338416    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338569    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338593    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338605    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338613    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338622    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338630    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpt8f\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-kube-api-access-jpt8f\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.338741 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.338643    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca995be6-88c2-474e-ab02-7b6fa35f4c9c-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.364235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.363851    5033 patch_prober.go:29] interesting pod/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:32:52.364235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.363906    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:32:52.461141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.460826    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:52.543368 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.543150    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\") pod \"16155a9f-35a5-4736-ac03-95a3a8743068\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") "
Oct 30 19:32:52.543543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.543427    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\") pod \"16155a9f-35a5-4736-ac03-95a3a8743068\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") "
Oct 30 19:32:52.543612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.543549    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\" (UniqueName: \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\") pod \"16155a9f-35a5-4736-ac03-95a3a8743068\" (UID: \"16155a9f-35a5-4736-ac03-95a3a8743068\") "
Oct 30 19:32:52.545502 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.545142    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca" pod "16155a9f-35a5-4736-ac03-95a3a8743068" (UID: "16155a9f-35a5-4736-ac03-95a3a8743068"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:52.547907 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.547446    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:32:52.554154 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.553565    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca" pod "16155a9f-35a5-4736-ac03-95a3a8743068" (UID: "16155a9f-35a5-4736-ac03-95a3a8743068"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:52.558498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.558075    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-d55178-openshift-default-5c669c9fdf-kz9rz"]
Oct 30 19:32:52.564710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.564591    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj" pod "16155a9f-35a5-4736-ac03-95a3a8743068" (UID: "16155a9f-35a5-4736-ac03-95a3a8743068"). InnerVolumeSpecName "kube-api-access-xp8sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.658323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.658113    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.658323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.658155    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp8sj\" (UniqueName: \"kubernetes.io/projected/16155a9f-35a5-4736-ac03-95a3a8743068-kube-api-access-xp8sj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.658323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.658173    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16155a9f-35a5-4736-ac03-95a3a8743068-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.783854 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.781900    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:52.785429 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.785405    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:52.864979 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.864941    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865034    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865110    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\" (UniqueName: \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.865186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865144    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865178    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\" (UniqueName: \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865220    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\" (UniqueName: \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.865417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865259    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.865417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865297    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.865417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865369    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.865619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865437    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865530    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\") pod \"e83d1981-e352-46a7-9cc3-a373943ab3d3\" (UID: \"e83d1981-e352-46a7-9cc3-a373943ab3d3\") "
Oct 30 19:32:52.865619 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.865581    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\") pod \"34d2be18-505a-416f-aec5-b0363d0f75d4\" (UID: \"34d2be18-505a-416f-aec5-b0363d0f75d4\") "
Oct 30 19:32:52.866380 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.866351    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.866589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.866569    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.866870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.866842    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.867653 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.867374    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.868548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.868512    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.868806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.868772    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.899390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.899359    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:52.899390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.899381    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "kube-api-access-4cvkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.899576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.899366    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm" pod "e83d1981-e352-46a7-9cc3-a373943ab3d3" (UID: "e83d1981-e352-46a7-9cc3-a373943ab3d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.899576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.899502    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "kube-api-access-bxk9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:52.899576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.899503    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:52.901346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.901316    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm" pod "34d2be18-505a-416f-aec5-b0363d0f75d4" (UID: "34d2be18-505a-416f-aec5-b0363d0f75d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:52.917356 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.913397    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing" containerID="cri-o://30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f" gracePeriod=25
Oct 30 19:32:52.967637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967596    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.967637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967638    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967654    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967668    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967699    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxk9j\" (UniqueName: \"kubernetes.io/projected/34d2be18-505a-416f-aec5-b0363d0f75d4-kube-api-access-bxk9j\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967715    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d1981-e352-46a7-9cc3-a373943ab3d3-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967730    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cvkj\" (UniqueName: \"kubernetes.io/projected/e83d1981-e352-46a7-9cc3-a373943ab3d3-kube-api-access-4cvkj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967745    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34d2be18-505a-416f-aec5-b0363d0f75d4-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967757    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967769    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967780    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34d2be18-505a-416f-aec5-b0363d0f75d4-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:52.968190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:52.967792    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d1981-e352-46a7-9cc3-a373943ab3d3-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:53.174956 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.174874    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" path="/var/lib/kubelet/pods/4d0be903-95d1-475a-9c0c-d32b252c81a0/volumes"
Oct 30 19:32:53.175417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.175400    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" path="/var/lib/kubelet/pods/ca995be6-88c2-474e-ab02-7b6fa35f4c9c/volumes"
Oct 30 19:32:53.176196 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.176168    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:32:53.187390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.187361    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:32:53.221909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.221880    5033 generic.go:356] "Generic (PLEG): container finished" podID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerID="1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e" exitCode=0
Oct 30 19:32:53.222063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.221933    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"
Oct 30 19:32:53.222063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.221965    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" event={"ID":"e83d1981-e352-46a7-9cc3-a373943ab3d3","Type":"ContainerDied","Data":"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"}
Oct 30 19:32:53.222063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.222011    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68" event={"ID":"e83d1981-e352-46a7-9cc3-a373943ab3d3","Type":"ContainerDied","Data":"bb8b7cfcb787c5031840d37a39cd991daa8ab2bb204ca8f064c8c918fe2ed9f2"}
Oct 30 19:32:53.222063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.222035    5033 scope.go:122] "RemoveContainer" containerID="1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"
Oct 30 19:32:53.223503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.223483    5033 generic.go:356] "Generic (PLEG): container finished" podID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerID="2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f" exitCode=0
Oct 30 19:32:53.223616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.223526    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"
Oct 30 19:32:53.223616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.223561    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" event={"ID":"34d2be18-505a-416f-aec5-b0363d0f75d4","Type":"ContainerDied","Data":"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"}
Oct 30 19:32:53.223616 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.223593    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq" event={"ID":"34d2be18-505a-416f-aec5-b0363d0f75d4","Type":"ContainerDied","Data":"0bf6288d87e45bb582607d474b8693a530e1febf6df0c479e1fdb7c33c507df0"}
Oct 30 19:32:53.225858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.225835    5033 generic.go:356] "Generic (PLEG): container finished" podID="0a607509-8147-453f-a594-09a247f6ced5" containerID="90bb24e3d861ceb6032c4fad546c74b796bbef34537191f96e5fa4acb8980db1" exitCode=0
Oct 30 19:32:53.225996 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.225980    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7" event={"ID":"0a607509-8147-453f-a594-09a247f6ced5","Type":"ContainerDied","Data":"90bb24e3d861ceb6032c4fad546c74b796bbef34537191f96e5fa4acb8980db1"}
Oct 30 19:32:53.227959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.227934    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"
Oct 30 19:32:53.228055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.227981    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj" event={"ID":"16155a9f-35a5-4736-ac03-95a3a8743068","Type":"ContainerDied","Data":"357de5548de0225711bd23b2c87df4637ce85d6d18af4497302899b6eedeceb3"}
Oct 30 19:32:53.228055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.228022    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="357de5548de0225711bd23b2c87df4637ce85d6d18af4497302899b6eedeceb3"
Oct 30 19:32:53.234310 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.234285    5033 scope.go:122] "RemoveContainer" containerID="1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"
Oct 30 19:32:53.234580 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:53.234560    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e\": container with ID starting with 1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e not found: ID does not exist" containerID="1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"
Oct 30 19:32:53.234649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.234588    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e"} err="failed to get container status \"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e\": rpc error: code = NotFound desc = could not find container \"1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e\": container with ID starting with 1c20534588a04702db4cb095338e4043e35d8d721429cb9ebab042043bb6593e not found: ID does not exist"
Oct 30 19:32:53.234649 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.234606    5033 scope.go:122] "RemoveContainer" containerID="2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"
Oct 30 19:32:53.244089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.244068    5033 scope.go:122] "RemoveContainer" containerID="2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"
Oct 30 19:32:53.244376 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:53.244357    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f\": container with ID starting with 2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f not found: ID does not exist" containerID="2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"
Oct 30 19:32:53.244451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.244384    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f"} err="failed to get container status \"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f\": rpc error: code = NotFound desc = could not find container \"2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f\": container with ID starting with 2865c20c4ba523d72804e5284728153b4603409b6364ac69e8ada3cc5501944f not found: ID does not exist"
Oct 30 19:32:53.255042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.255023    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:32:53.260545 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.260524    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/distinct-2-w6-639c15-kserve-7479c875bf-g8jgq"]
Oct 30 19:32:53.274158 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.274124    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:32:53.279637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:53.279613    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-639c15/simulator-w6-639c15-kserve-79489b87f8-6ph68"]
Oct 30 19:32:54.456623 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.456589    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:54.590540 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.590464    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\" (UniqueName: \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\") pod \"0a607509-8147-453f-a594-09a247f6ced5\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") "
Oct 30 19:32:54.591531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.591507    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\") pod \"0a607509-8147-453f-a594-09a247f6ced5\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") "
Oct 30 19:32:54.592266 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.592245    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\") pod \"0a607509-8147-453f-a594-09a247f6ced5\" (UID: \"0a607509-8147-453f-a594-09a247f6ced5\") "
Oct 30 19:32:54.594028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.593753    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg" pod "0a607509-8147-453f-a594-09a247f6ced5" (UID: "0a607509-8147-453f-a594-09a247f6ced5"). InnerVolumeSpecName "kube-api-access-v7jbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:54.594583 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.594561    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7jbg\" (UniqueName: \"kubernetes.io/projected/0a607509-8147-453f-a594-09a247f6ced5-kube-api-access-v7jbg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.594749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.594728    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca" pod "0a607509-8147-453f-a594-09a247f6ced5" (UID: "0a607509-8147-453f-a594-09a247f6ced5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:54.596403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.596379    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca" pod "0a607509-8147-453f-a594-09a247f6ced5" (UID: "0a607509-8147-453f-a594-09a247f6ced5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:54.695919 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.695883    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a607509-8147-453f-a594-09a247f6ced5-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.696029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.695922    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a607509-8147-453f-a594-09a247f6ced5-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.985209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:54.982757    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:32:55.122085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.119500    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.122085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.119580    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.122085 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.119654    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.122749 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.122703    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.122871    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.122934    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.123008    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.123083    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.123117    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\") pod \"901b2c57-dae5-418e-9273-4701dfbcaa26\" (UID: \"901b2c57-dae5-418e-9273-4701dfbcaa26\") "
Oct 30 19:32:55.124667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.124463    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.125126 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.124802    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.127857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.127620    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.128194 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.128012    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.128640 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.128355    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.159608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.154868    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:55.202663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.199242    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" path="/var/lib/kubelet/pods/34d2be18-505a-416f-aec5-b0363d0f75d4/volumes"
Oct 30 19:32:55.202663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.200031    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" path="/var/lib/kubelet/pods/e83d1981-e352-46a7-9cc3-a373943ab3d3/volumes"
Oct 30 19:32:55.216152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.215713    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:55.218161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.218127    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.223752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.223720    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq" pod "901b2c57-dae5-418e-9273-4701dfbcaa26" (UID: "901b2c57-dae5-418e-9273-4701dfbcaa26"). InnerVolumeSpecName "kube-api-access-zpwxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225603    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225634    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225645    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225654    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225662    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/901b2c57-dae5-418e-9273-4701dfbcaa26-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225671    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/901b2c57-dae5-418e-9273-4701dfbcaa26-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225727    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225735    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/901b2c57-dae5-418e-9273-4701dfbcaa26-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.225772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.225745    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpwxq\" (UniqueName: \"kubernetes.io/projected/901b2c57-dae5-418e-9273-4701dfbcaa26-kube-api-access-zpwxq\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.226779 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.226759    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:32:55.259556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.259516    5033 generic.go:356] "Generic (PLEG): container finished" podID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerID="f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962" exitCode=0
Oct 30 19:32:55.259726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.259703    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" event={"ID":"901b2c57-dae5-418e-9273-4701dfbcaa26","Type":"ContainerDied","Data":"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"}
Oct 30 19:32:55.259803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.259756    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld" event={"ID":"901b2c57-dae5-418e-9273-4701dfbcaa26","Type":"ContainerDied","Data":"055a9a22ff0462f8c31e0f00650c0a73e0db3db2f66d993c5ca6ced7b24c80d2"}
Oct 30 19:32:55.259803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.259785    5033 scope.go:122] "RemoveContainer" containerID="f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"
Oct 30 19:32:55.260590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.260567    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"
Oct 30 19:32:55.265209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.264758    5033 generic.go:356] "Generic (PLEG): container finished" podID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerID="cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8" exitCode=0
Oct 30 19:32:55.265209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.264957    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" event={"ID":"ca0b452a-f907-42ec-abe5-8fad195a14c3","Type":"ContainerDied","Data":"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"}
Oct 30 19:32:55.265209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.265000    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8" event={"ID":"ca0b452a-f907-42ec-abe5-8fad195a14c3","Type":"ContainerDied","Data":"4e5eeed3faf5ac92cffbdbd563ccc3cdcc37eae89fb912714d14a54d13a97533"}
Oct 30 19:32:55.265209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.265068    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"
Oct 30 19:32:55.295719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.294015    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7" event={"ID":"0a607509-8147-453f-a594-09a247f6ced5","Type":"ContainerDied","Data":"84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694"}
Oct 30 19:32:55.295719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.294074    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84c834836c40833349c715573689367142f10d1e0cfdb07e043b44ccf254e694"
Oct 30 19:32:55.295719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.294075    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"
Oct 30 19:32:55.326191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.326162    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.329173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.329147    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.329395 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.329379    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.329487 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.329462    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.329593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.329579    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.329788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.329771    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.330214 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330198    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.330384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330370    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.330627 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330610    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.331077 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.331055    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\") pod \"ca0b452a-f907-42ec-abe5-8fad195a14c3\" (UID: \"ca0b452a-f907-42ec-abe5-8fad195a14c3\") "
Oct 30 19:32:55.332259 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.332237    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.332413 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330093    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.332522 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330473    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.332613 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.330803    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.333351 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.331425    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.333492 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.333153    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "kube-api-access-zdr5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:55.334039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.333302    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:55.352292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.349765    5033 scope.go:122] "RemoveContainer" containerID="f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"
Oct 30 19:32:55.352292 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:55.350152    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962\": container with ID starting with f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962 not found: ID does not exist" containerID="f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"
Oct 30 19:32:55.352292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.350187    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962"} err="failed to get container status \"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962\": rpc error: code = NotFound desc = could not find container \"f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962\": container with ID starting with f5b5d82d8efdff8a901f519cea4c539563e640fd3e142c2e2ed1db6e9a1d1962 not found: ID does not exist"
Oct 30 19:32:55.352292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.350210    5033 scope.go:122] "RemoveContainer" containerID="cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"
Oct 30 19:32:55.352292 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.352146    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.355367 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.355329    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token" pod "ca0b452a-f907-42ec-abe5-8fad195a14c3" (UID: "ca0b452a-f907-42ec-abe5-8fad195a14c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:55.378879 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.378830    5033 scope.go:122] "RemoveContainer" containerID="cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"
Oct 30 19:32:55.379854 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:32:55.379815    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8\": container with ID starting with cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8 not found: ID does not exist" containerID="cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"
Oct 30 19:32:55.379966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.379865    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8"} err="failed to get container status \"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8\": rpc error: code = NotFound desc = could not find container \"cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8\": container with ID starting with cc588d2820e2af8e69a241914d683958844d4e4e986d58cecc3d03b38cbbdea8 not found: ID does not exist"
Oct 30 19:32:55.387949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.387916    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:32:55.407752 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.406377    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-wdfld"]
Oct 30 19:32:55.466363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.466306    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.466869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.466847    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0b452a-f907-42ec-abe5-8fad195a14c3-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.466958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.466947    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.467026 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.467016    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.467090 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.467081    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.467157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.467148    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.471499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.467223    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdr5v\" (UniqueName: \"kubernetes.io/projected/ca0b452a-f907-42ec-abe5-8fad195a14c3-kube-api-access-zdr5v\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.471499 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.467239    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0b452a-f907-42ec-abe5-8fad195a14c3-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.619431 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.619401    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:32:55.626078 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:55.626056    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9f2d57-openshift-default-79bc845645-45zx8"]
Oct 30 19:32:57.171446 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.171413    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" path="/var/lib/kubelet/pods/901b2c57-dae5-418e-9273-4701dfbcaa26/volumes"
Oct 30 19:32:57.171988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.171972    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" path="/var/lib/kubelet/pods/ca0b452a-f907-42ec-abe5-8fad195a14c3/volumes"
Oct 30 19:32:57.941600 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.935559    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:32:57.941600 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.935927    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy" containerID="cri-o://8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452" gracePeriod=30
Oct 30 19:32:57.983688 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.983650    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:32:57.983915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.983886    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy" containerID="cri-o://034bab3d26dc39f031e1391260eee6a407119bac1593d7f3bb05b6171331c60c" gracePeriod=30
Oct 30 19:32:57.990043 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.990014    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:32:57.990202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:57.990184    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy" containerID="cri-o://4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f" gracePeriod=30
Oct 30 19:32:59.336949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.336847    5033 generic.go:356] "Generic (PLEG): container finished" podID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerID="60b83c5026f0cc9bf0504a8fa2ca694e7cf3b48db88f3f7fc3e57eec19efec2c" exitCode=0
Oct 30 19:32:59.337384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.337250    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" event={"ID":"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34","Type":"ContainerDied","Data":"60b83c5026f0cc9bf0504a8fa2ca694e7cf3b48db88f3f7fc3e57eec19efec2c"}
Oct 30 19:32:59.339282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.339173    5033 generic.go:356] "Generic (PLEG): container finished" podID="904ebe95-7d4f-4d65-a224-98c169f90681" containerID="47160548286b7a060863053e28d8386e9cd5cd228140c044c6aac3be6f00bf9f" exitCode=0
Oct 30 19:32:59.339282 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.339226    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" event={"ID":"904ebe95-7d4f-4d65-a224-98c169f90681","Type":"ContainerDied","Data":"47160548286b7a060863053e28d8386e9cd5cd228140c044c6aac3be6f00bf9f"}
Oct 30 19:32:59.372964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.372937    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:32:59.427982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.427954    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:32:59.434402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.434375    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\" (UniqueName: \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\") pod \"904ebe95-7d4f-4d65-a224-98c169f90681\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") "
Oct 30 19:32:59.434528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.434424    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\") pod \"904ebe95-7d4f-4d65-a224-98c169f90681\" (UID: \"904ebe95-7d4f-4d65-a224-98c169f90681\") "
Oct 30 19:32:59.434950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.434925    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume" pod "904ebe95-7d4f-4d65-a224-98c169f90681" (UID: "904ebe95-7d4f-4d65-a224-98c169f90681"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:59.435103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.435078    5033 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904ebe95-7d4f-4d65-a224-98c169f90681-plugins-config-volume\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:59.437182 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.437155    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h" pod "904ebe95-7d4f-4d65-a224-98c169f90681" (UID: "904ebe95-7d4f-4d65-a224-98c169f90681"). InnerVolumeSpecName "kube-api-access-x7m6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:59.536587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.536556    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\") pod \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") "
Oct 30 19:32:59.536779 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.536740    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\" (UniqueName: \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\") pod \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\" (UID: \"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34\") "
Oct 30 19:32:59.536893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.536872    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume" pod "99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" (UID: "99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:59.537157 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.537142    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7m6h\" (UniqueName: \"kubernetes.io/projected/904ebe95-7d4f-4d65-a224-98c169f90681-kube-api-access-x7m6h\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:59.537223 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.537157    5033 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-plugins-config-volume\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:59.538738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.538719    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw" pod "99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" (UID: "99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34"). InnerVolumeSpecName "kube-api-access-vxqlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:59.618927 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.618798    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"]
Oct 30 19:32:59.620191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620167    5033 cpu_manager.go:414] "containerMap: removing container" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy"
Oct 30 19:32:59.620191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620191    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620221    5033 cpu_manager.go:414] "containerMap: removing container" podUID="0a607509-8147-453f-a594-09a247f6ced5" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620230    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a607509-8147-453f-a594-09a247f6ced5" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620285    5033 cpu_manager.go:414] "containerMap: removing container" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620294    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620330    5033 cpu_manager.go:414] "containerMap: removing container" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620338    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620384    5033 cpu_manager.go:414] "containerMap: removing container" podUID="16155a9f-35a5-4736-ac03-95a3a8743068" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620391    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16155a9f-35a5-4736-ac03-95a3a8743068" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620414    5033 cpu_manager.go:414] "containerMap: removing container" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620427    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620440    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620449    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620469    5033 cpu_manager.go:414] "containerMap: removing container" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620477    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620503    5033 cpu_manager.go:414] "containerMap: removing container" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620510    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620529    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620537    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620562    5033 cpu_manager.go:414] "containerMap: removing container" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620569    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620851    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="e83d1981-e352-46a7-9cc3-a373943ab3d3" containerName="main"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620869    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620892    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca995be6-88c2-474e-ab02-7b6fa35f4c9c" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620910    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca0b452a-f907-42ec-abe5-8fad195a14c3" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620920    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a607509-8147-453f-a594-09a247f6ced5" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620933    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" containerName="payload-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620952    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="901b2c57-dae5-418e-9273-4701dfbcaa26" containerName="istio-proxy"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620975    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="16155a9f-35a5-4736-ac03-95a3a8743068" containerName="revoke-keys"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.620985    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" containerName="payload-pre-processing"
Oct 30 19:32:59.621063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.621002    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="34d2be18-505a-416f-aec5-b0363d0f75d4" containerName="main"
Oct 30 19:32:59.627142 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.627108    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.629283 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.629258    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"]
Oct 30 19:32:59.638731 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.638710    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxqlw\" (UniqueName: \"kubernetes.io/projected/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34-kube-api-access-vxqlw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:59.705438 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.705404    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:32:59.705438 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.705422    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:59.740416 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.740387    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.740544 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.740530    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqzlt\" (UniqueName: \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.740600 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.740588    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.841742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.841708    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqzlt\" (UniqueName: \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.841940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.841789    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.841940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.841872    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.842503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.842479    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.844547 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.844517    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.851228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.851200    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqzlt\" (UniqueName: \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\") pod \"maas-api-revoke-sub-1395101c-hr6ft\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:32:59.966719 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:32:59.964653    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:33:00.260759 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:33:00.260713    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aea349b_f447_472d_8e0a_a6e9ca9e96f6.slice/crio-b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07 WatchSource:0}: Error finding container b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07: Status 404 returned error can't find the container with id b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07
Oct 30 19:33:00.262181 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.262146    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"]
Oct 30 19:33:00.348684 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.348652    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft" event={"ID":"4aea349b-f447-472d-8e0a-a6e9ca9e96f6","Type":"ContainerStarted","Data":"b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07"}
Oct 30 19:33:00.350267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.350235    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"
Oct 30 19:33:00.350267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.350256    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt" event={"ID":"904ebe95-7d4f-4d65-a224-98c169f90681","Type":"ContainerDied","Data":"a4306138482d6d6502b031e15fdeab6f672b998303e15792c88d05ba07aa434c"}
Oct 30 19:33:00.350447 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.350312    5033 scope.go:122] "RemoveContainer" containerID="47160548286b7a060863053e28d8386e9cd5cd228140c044c6aac3be6f00bf9f"
Oct 30 19:33:00.352123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.352098    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx" event={"ID":"99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34","Type":"ContainerDied","Data":"344fc8cc5ee000d61e8b54f4705874b5b12d9abc4f4e95105f55866866f417a1"}
Oct 30 19:33:00.352262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.352239    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"
Oct 30 19:33:00.361823 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.361802    5033 scope.go:122] "RemoveContainer" containerID="60b83c5026f0cc9bf0504a8fa2ca694e7cf3b48db88f3f7fc3e57eec19efec2c"
Oct 30 19:33:00.378909 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.378885    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:33:00.385204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.385181    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-171808-667d897c7f-8f4kt"]
Oct 30 19:33:00.398054 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.398033    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:33:00.407235 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.407200    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-171808-67cc746dd9-mvmhx"]
Oct 30 19:33:00.458568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.458536    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"]
Oct 30 19:33:00.459417 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.459389    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d0be903-95d1-475a-9c0c-d32b252c81a0" containerName="istio-proxy"
Oct 30 19:33:00.472897 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.472874    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"]
Oct 30 19:33:00.473047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.472983    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.573850 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.573778    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.573850 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.573829    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv7c5\" (UniqueName: \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.574032 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.573965    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.675066 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.675035    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv7c5\" (UniqueName: \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.675192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.675180    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.675273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.675254    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.675936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.675913    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.677836 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.677810    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.686063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.686039    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv7c5\" (UniqueName: \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\") pod \"maas-api-revoke-sub-9d944f2b-hxhnf\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:00.806042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:00.806018    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:01.129892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.129862    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"]
Oct 30 19:33:01.130105 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:33:01.130079    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb58caf_9f9d_43b1_ac07_0f402058348e.slice/crio-4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225 WatchSource:0}: Error finding container 4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225: Status 404 returned error can't find the container with id 4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225
Oct 30 19:33:01.170838 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.170812    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="904ebe95-7d4f-4d65-a224-98c169f90681" path="/var/lib/kubelet/pods/904ebe95-7d4f-4d65-a224-98c169f90681/volumes"
Oct 30 19:33:01.171300 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.171279    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34" path="/var/lib/kubelet/pods/99cf5f80-c8f2-48ca-a4af-fe21cdb7dc34/volumes"
Oct 30 19:33:01.361994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.361961    5033 generic.go:356] "Generic (PLEG): container finished" podID="4aea349b-f447-472d-8e0a-a6e9ca9e96f6" containerID="c3deea7160bb623fc2606b8354bc54ca929414d4a46a52ca979710bcaca61cea" exitCode=0
Oct 30 19:33:01.362411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.362041    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft" event={"ID":"4aea349b-f447-472d-8e0a-a6e9ca9e96f6","Type":"ContainerDied","Data":"c3deea7160bb623fc2606b8354bc54ca929414d4a46a52ca979710bcaca61cea"}
Oct 30 19:33:01.363605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:01.363561    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf" event={"ID":"bdb58caf-9f9d-43b1-ac07-0f402058348e","Type":"ContainerStarted","Data":"4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225"}
Oct 30 19:33:02.102517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.102480    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:33:02.246905 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.246874    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:33:02.247118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.247093    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" podUID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" containerName="maas-api" containerID="cri-o://5e7348cbd9d8604f5ffc67c595330a346a198d712b84e77b202aafa38792c933" gracePeriod=30
Oct 30 19:33:02.410402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.407161    5033 generic.go:356] "Generic (PLEG): container finished" podID="bdb58caf-9f9d-43b1-ac07-0f402058348e" containerID="c4a78ffcbe64f32d14807962e281ca3c5a03054d09fad44eb153a12abf4f2b77" exitCode=0
Oct 30 19:33:02.410402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.407298    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf" event={"ID":"bdb58caf-9f9d-43b1-ac07-0f402058348e","Type":"ContainerDied","Data":"c4a78ffcbe64f32d14807962e281ca3c5a03054d09fad44eb153a12abf4f2b77"}
Oct 30 19:33:02.422633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.421264    5033 generic.go:356] "Generic (PLEG): container finished" podID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" containerID="5e7348cbd9d8604f5ffc67c595330a346a198d712b84e77b202aafa38792c933" exitCode=0
Oct 30 19:33:02.422633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.421920    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" event={"ID":"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda","Type":"ContainerDied","Data":"5e7348cbd9d8604f5ffc67c595330a346a198d712b84e77b202aafa38792c933"}
Oct 30 19:33:02.861655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.861625    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:33:02.898958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.898882    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:33:02.937495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937454    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\") pod \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") "
Oct 30 19:33:02.937746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937537    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\") pod \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") "
Oct 30 19:33:02.937746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937618    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\") pod \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") "
Oct 30 19:33:02.937746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937650    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\") pod \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") "
Oct 30 19:33:02.937936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937750    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\" (UniqueName: \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\") pod \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") "
Oct 30 19:33:02.937936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937842    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\") pod \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\" (UID: \"4aea349b-f447-472d-8e0a-a6e9ca9e96f6\") "
Oct 30 19:33:02.937936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937885    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\" (UniqueName: \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\") pod \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\" (UID: \"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda\") "
Oct 30 19:33:02.938103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.937966    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca" pod "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" (UID: "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:02.938539 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.938514    5033 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-openshift-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:02.939173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.939147    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca" pod "4aea349b-f447-472d-8e0a-a6e9ca9e96f6" (UID: "4aea349b-f447-472d-8e0a-a6e9ca9e96f6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:02.941411 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.941351    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs" pod "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" (UID: "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:02.941798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.941765    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls" pod "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" (UID: "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:02.943740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.942745    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca" pod "4aea349b-f447-472d-8e0a-a6e9ca9e96f6" (UID: "4aea349b-f447-472d-8e0a-a6e9ca9e96f6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:02.943740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.942774    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62" pod "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" (UID: "b84a3a0a-1edc-4e03-a4a5-1e7862b14bda"). InnerVolumeSpecName "kube-api-access-twn62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:02.943740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:02.943642    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt" pod "4aea349b-f447-472d-8e0a-a6e9ca9e96f6" (UID: "4aea349b-f447-472d-8e0a-a6e9ca9e96f6"). InnerVolumeSpecName "kube-api-access-zqzlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.039668 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039637    5033 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-metrics-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.039668 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039669    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.039892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039722    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-maas-api-tls\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.039892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039736    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqzlt\" (UniqueName: \"kubernetes.io/projected/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-kube-api-access-zqzlt\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.039892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039749    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aea349b-f447-472d-8e0a-a6e9ca9e96f6-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.039892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.039761    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twn62\" (UniqueName: \"kubernetes.io/projected/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda-kube-api-access-twn62\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.328314 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.327940    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:33:03.434524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.434373    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw" event={"ID":"b84a3a0a-1edc-4e03-a4a5-1e7862b14bda","Type":"ContainerDied","Data":"fa286e0ec55bb40bde92725e739b7708f841afded077e2dffbbd5a669dcc28f8"}
Oct 30 19:33:03.434524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.434403    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"
Oct 30 19:33:03.434524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.434443    5033 scope.go:122] "RemoveContainer" containerID="5e7348cbd9d8604f5ffc67c595330a346a198d712b84e77b202aafa38792c933"
Oct 30 19:33:03.439162 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.438836    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:33:03.441048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.440918    5033 generic.go:356] "Generic (PLEG): container finished" podID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerID="034bab3d26dc39f031e1391260eee6a407119bac1593d7f3bb05b6171331c60c" exitCode=0
Oct 30 19:33:03.441048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.440944    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" event={"ID":"a27fa030-73b4-4864-8aaf-29bc3451860d","Type":"ContainerDied","Data":"034bab3d26dc39f031e1391260eee6a407119bac1593d7f3bb05b6171331c60c"}
Oct 30 19:33:03.441048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.440978    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c" event={"ID":"a27fa030-73b4-4864-8aaf-29bc3451860d","Type":"ContainerDied","Data":"2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0"}
Oct 30 19:33:03.441048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.440993    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2613a0095602eabf6b6ebfa94ed2d6b961851f346bb5541687c182f23e8372c0"
Oct 30 19:33:03.443577 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.443550    5033 generic.go:356] "Generic (PLEG): container finished" podID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerID="8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452" exitCode=0
Oct 30 19:33:03.443704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.443665    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"
Oct 30 19:33:03.443835 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.443814    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" event={"ID":"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9","Type":"ContainerDied","Data":"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"}
Oct 30 19:33:03.443958 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.443856    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk" event={"ID":"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9","Type":"ContainerDied","Data":"ecc83d3ad911f808242e40feacd1a145f5735cd3f53e81048db617798e910029"}
Oct 30 19:33:03.445472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.445453    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:33:03.445861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.445841    5033 generic.go:356] "Generic (PLEG): container finished" podID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerID="4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f" exitCode=0
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446225    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446256    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" event={"ID":"2fde99e3-88c5-44d9-bc8c-20173f17f42b","Type":"ContainerDied","Data":"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"}
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446337    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q" event={"ID":"2fde99e3-88c5-44d9-bc8c-20173f17f42b","Type":"ContainerDied","Data":"f216130d2ad1be83e3a072dd7d26b0ee5ca4eba1213630f50dbea0b255509c32"}
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446664    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446785    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446844    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446922    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.446959    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.447013    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.447114    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.447198    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.447330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.447242    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\") pod \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\" (UID: \"fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9\") "
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.447010    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.448078    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.449239    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.449426    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.449845    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.450338    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft" event={"ID":"4aea349b-f447-472d-8e0a-a6e9ca9e96f6","Type":"ContainerDied","Data":"b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07"}
Oct 30 19:33:03.450642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.450371    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3eb9fc9052f9ada27d5910e8c2b72d1d07fb1495aa4cab9fef2990fb2219e07"
Oct 30 19:33:03.451382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.451109    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:03.451382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.451171    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:03.451610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.451574    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.453696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.453648    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.457565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.457541    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc" pod "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" (UID: "fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9"). InnerVolumeSpecName "kube-api-access-rq4xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.460281 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.460255    5033 scope.go:122] "RemoveContainer" containerID="8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"
Oct 30 19:33:03.481648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.480463    5033 scope.go:122] "RemoveContainer" containerID="8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"
Oct 30 19:33:03.481648 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:03.480941    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452\": container with ID starting with 8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452 not found: ID does not exist" containerID="8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"
Oct 30 19:33:03.481648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.480978    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452"} err="failed to get container status \"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452\": rpc error: code = NotFound desc = could not find container \"8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452\": container with ID starting with 8eccb1a3ce82e9868e8d084ee8405b19c2e901ec1fba6db567a8b2ff94947452 not found: ID does not exist"
Oct 30 19:33:03.481648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.481002    5033 scope.go:122] "RemoveContainer" containerID="4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"
Oct 30 19:33:03.489781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.489757    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:33:03.496249 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.496223    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-639c15-7688ff4654-prplw"]
Oct 30 19:33:03.508991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.508897    5033 scope.go:122] "RemoveContainer" containerID="4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"
Oct 30 19:33:03.509263 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:03.509205    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f\": container with ID starting with 4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f not found: ID does not exist" containerID="4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"
Oct 30 19:33:03.509263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.509233    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f"} err="failed to get container status \"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f\": rpc error: code = NotFound desc = could not find container \"4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f\": container with ID starting with 4a2e44495451c4c0b0d4ad50cddbd5e8acb47a343e7af6795cece082b1192c6f not found: ID does not exist"
Oct 30 19:33:03.548516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548429    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.548516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548474    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548542    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548581    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548616    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548732    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548772    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548803    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548831    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548888    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548937    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.548965    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549040    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549145    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549196    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549258    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549314    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\") pod \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\" (UID: \"2fde99e3-88c5-44d9-bc8c-20173f17f42b\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549378    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.549445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549411    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\") pod \"a27fa030-73b4-4864-8aaf-29bc3451860d\" (UID: \"a27fa030-73b4-4864-8aaf-29bc3451860d\") "
Oct 30 19:33:03.550315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.549783    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.550377    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.550624    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551225    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551557    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551578    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551595    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551624    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551642    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551656    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551576    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551729    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551748    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq4xc\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-kube-api-access-rq4xc\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551916    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551929    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551942    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551956    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551987    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.551999    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.552989    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.553067    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.553117    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.553191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.553169    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.559154 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.559128    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "kube-api-access-cs7cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.559464 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.559431    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.559575 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.559560    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:03.559791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.559747    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:03.559991 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.559941    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:03.560347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.560319    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.560516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.560504    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7" pod "a27fa030-73b4-4864-8aaf-29bc3451860d" (UID: "a27fa030-73b4-4864-8aaf-29bc3451860d"). InnerVolumeSpecName "kube-api-access-4rfr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.562342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.562318    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token" pod "2fde99e3-88c5-44d9-bc8c-20173f17f42b" (UID: "2fde99e3-88c5-44d9-bc8c-20173f17f42b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.648853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.648822    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:03.653339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653316    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653339 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653338    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653351    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rfr7\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-kube-api-access-4rfr7\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653361    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653371    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs7cg\" (UniqueName: \"kubernetes.io/projected/2fde99e3-88c5-44d9-bc8c-20173f17f42b-kube-api-access-cs7cg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653381    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653390    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653398    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653406    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fde99e3-88c5-44d9-bc8c-20173f17f42b-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653414    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653422    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fde99e3-88c5-44d9-bc8c-20173f17f42b-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653430    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27fa030-73b4-4864-8aaf-29bc3451860d-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.653541 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.653438    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27fa030-73b4-4864-8aaf-29bc3451860d-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.753929 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.753897    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\" (UniqueName: \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\") pod \"bdb58caf-9f9d-43b1-ac07-0f402058348e\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") "
Oct 30 19:33:03.754107 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.754093    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\") pod \"bdb58caf-9f9d-43b1-ac07-0f402058348e\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") "
Oct 30 19:33:03.754205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.754193    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\") pod \"bdb58caf-9f9d-43b1-ac07-0f402058348e\" (UID: \"bdb58caf-9f9d-43b1-ac07-0f402058348e\") "
Oct 30 19:33:03.754604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.754576    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca" pod "bdb58caf-9f9d-43b1-ac07-0f402058348e" (UID: "bdb58caf-9f9d-43b1-ac07-0f402058348e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:03.754763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.754743    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.758525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.756443    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca" pod "bdb58caf-9f9d-43b1-ac07-0f402058348e" (UID: "bdb58caf-9f9d-43b1-ac07-0f402058348e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:03.758525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.756564    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5" pod "bdb58caf-9f9d-43b1-ac07-0f402058348e" (UID: "bdb58caf-9f9d-43b1-ac07-0f402058348e"). InnerVolumeSpecName "kube-api-access-hv7c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:03.773760 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.773735    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:33:03.778269 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.778244    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-zkjjk"]
Oct 30 19:33:03.790501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.790479    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:33:03.798072 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.798050    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-65r9q"]
Oct 30 19:33:03.855949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.855916    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv7c5\" (UniqueName: \"kubernetes.io/projected/bdb58caf-9f9d-43b1-ac07-0f402058348e-kube-api-access-hv7c5\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:03.855949 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:03.855948    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bdb58caf-9f9d-43b1-ac07-0f402058348e-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:04.460816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.460787    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"
Oct 30 19:33:04.461403 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.461375    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf" event={"ID":"bdb58caf-9f9d-43b1-ac07-0f402058348e","Type":"ContainerDied","Data":"4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225"}
Oct 30 19:33:04.461531 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.461430    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ce1b6613ce1da6a911ffb7da465a5ba7a466dd8b9a36472ef99f9cbdf4b4225"
Oct 30 19:33:04.465777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.465759    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"
Oct 30 19:33:04.493874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.493851    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:33:04.501436 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:04.501416    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-171808-openshift-default-856d98c98c-dd87c"]
Oct 30 19:33:04.997482 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:04.997445    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:33:05.194025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:05.190199    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" path="/var/lib/kubelet/pods/2fde99e3-88c5-44d9-bc8c-20173f17f42b/volumes"
Oct 30 19:33:05.208729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:05.206830    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" path="/var/lib/kubelet/pods/a27fa030-73b4-4864-8aaf-29bc3451860d/volumes"
Oct 30 19:33:05.208729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:05.207491    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" path="/var/lib/kubelet/pods/b84a3a0a-1edc-4e03-a4a5-1e7862b14bda/volumes"
Oct 30 19:33:05.208729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:05.208135    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" path="/var/lib/kubelet/pods/fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9/volumes"
Oct 30 19:33:06.012872 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.012784    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main" containerID="cri-o://76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67" gracePeriod=45
Oct 30 19:33:06.016895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.016860    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main" containerID="cri-o://874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7" gracePeriod=45
Oct 30 19:33:06.307116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.307067    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:33:06.358587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358559    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\" (UniqueName: \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.358744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358649    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.358744 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358702    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.358881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358824    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.358881 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358860    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.358980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.358898    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\" (UniqueName: \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\") pod \"aae483f3-d484-4f73-8ec4-dfde145ea837\" (UID: \"aae483f3-d484-4f73-8ec4-dfde145ea837\") "
Oct 30 19:33:06.359111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.359089    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.359538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.359513    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.360185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.360159    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.360654 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.360622    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.363261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.361995    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:06.363261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.362894    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "kube-api-access-pscdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:06.367348 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.367320    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:33:06.368089 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.368055    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm" pod "aae483f3-d484-4f73-8ec4-dfde145ea837" (UID: "aae483f3-d484-4f73-8ec4-dfde145ea837"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460014    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460094    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\" (UniqueName: \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460243    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460298    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460376    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460478    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.460847    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\" (UniqueName: \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\") pod \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\" (UID: \"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5\") "
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461416    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461437    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461452    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aae483f3-d484-4f73-8ec4-dfde145ea837-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461466    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461481    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pscdw\" (UniqueName: \"kubernetes.io/projected/aae483f3-d484-4f73-8ec4-dfde145ea837-kube-api-access-pscdw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461497    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aae483f3-d484-4f73-8ec4-dfde145ea837-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.461904    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.467727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.462143    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.477121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.477037    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:06.477121 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.477082    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "kube-api-access-bmfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:06.478558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.478541    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm" pod "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" (UID: "c01c0ecd-311a-4c1b-8d8a-af3e8debaef5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:06.488936 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.488907    5033 generic.go:356] "Generic (PLEG): container finished" podID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerID="874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7" exitCode=0
Oct 30 19:33:06.489065 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.489048    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" event={"ID":"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5","Type":"ContainerDied","Data":"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"}
Oct 30 19:33:06.489113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.489087    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w" event={"ID":"c01c0ecd-311a-4c1b-8d8a-af3e8debaef5","Type":"ContainerDied","Data":"d8846157434df8b594321971f4da960f05891f4a1de9e778aeecded588b6b76e"}
Oct 30 19:33:06.489113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.489109    5033 scope.go:122] "RemoveContainer" containerID="874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"
Oct 30 19:33:06.489261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.489249    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"
Oct 30 19:33:06.497704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.493715    5033 generic.go:356] "Generic (PLEG): container finished" podID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerID="76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67" exitCode=0
Oct 30 19:33:06.497704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.493749    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" event={"ID":"aae483f3-d484-4f73-8ec4-dfde145ea837","Type":"ContainerDied","Data":"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"}
Oct 30 19:33:06.497704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.493786    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph" event={"ID":"aae483f3-d484-4f73-8ec4-dfde145ea837","Type":"ContainerDied","Data":"d6055af97c768c848cf42b8dacda7895fecd1faf63ba5a981d0284b878365af2"}
Oct 30 19:33:06.497704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.493753    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"
Oct 30 19:33:06.506757 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.506734    5033 scope.go:122] "RemoveContainer" containerID="874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"
Oct 30 19:33:06.507104 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:06.507072    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7\": container with ID starting with 874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7 not found: ID does not exist" containerID="874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"
Oct 30 19:33:06.507204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.507113    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7"} err="failed to get container status \"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7\": rpc error: code = NotFound desc = could not find container \"874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7\": container with ID starting with 874bba065e708cd05f73a75e0318fa87e012a91c48789161406b25cdc450b3c7 not found: ID does not exist"
Oct 30 19:33:06.507204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.507136    5033 scope.go:122] "RemoveContainer" containerID="76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"
Oct 30 19:33:06.516963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.516941    5033 scope.go:122] "RemoveContainer" containerID="76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"
Oct 30 19:33:06.517229 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:06.517210    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67\": container with ID starting with 76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67 not found: ID does not exist" containerID="76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"
Oct 30 19:33:06.517295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.517239    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67"} err="failed to get container status \"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67\": rpc error: code = NotFound desc = could not find container \"76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67\": container with ID starting with 76b5aa27a76c3e76e826576039a931617c2b67f4a9f741b1215832b01874ee67 not found: ID does not exist"
Oct 30 19:33:06.531841 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.531732    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:33:06.536084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.536059    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-w5-f7f075-kserve-766cc8fbd9-hk59w"]
Oct 30 19:33:06.558354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.558331    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:33:06.562696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.562652    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmfn8\" (UniqueName: \"kubernetes.io/projected/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-kube-api-access-bmfn8\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.562696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.562694    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.562830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.562710    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.562830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.562726    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.562830 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.562739    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:06.563983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:06.563963    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f7f075/distinct-2-w5-f7f075-kserve-64989fc678-nkrph"]
Oct 30 19:33:07.179576 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:07.179536    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" path="/var/lib/kubelet/pods/aae483f3-d484-4f73-8ec4-dfde145ea837/volumes"
Oct 30 19:33:07.180959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:07.180930    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" path="/var/lib/kubelet/pods/c01c0ecd-311a-4c1b-8d8a-af3e8debaef5/volumes"
Oct 30 19:33:08.207217 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.207166    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main" containerID="cri-o://8e571525f9002409f6b3780758b3d7c92d053b0aebe49c5d42dffaa86ea7420d" gracePeriod=45
Oct 30 19:33:08.212186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.212011    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" containerID="cri-o://e45a7e9235e9b321a036b92047806c06b74a059fd5423fbcccc39d7c675fd8d6" gracePeriod=45
Oct 30 19:33:08.241028 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.241001    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": dial tcp 10.132.0.68:8000: connect: connection refused"
Oct 30 19:33:08.518059 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.518000    5033 generic.go:356] "Generic (PLEG): container finished" podID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerID="e45a7e9235e9b321a036b92047806c06b74a059fd5423fbcccc39d7c675fd8d6" exitCode=0
Oct 30 19:33:08.518254 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.518070    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" event={"ID":"ae35af12-8cd7-47bb-8031-0bfd1412b590","Type":"ContainerDied","Data":"e45a7e9235e9b321a036b92047806c06b74a059fd5423fbcccc39d7c675fd8d6"}
Oct 30 19:33:08.520015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.519565    5033 generic.go:356] "Generic (PLEG): container finished" podID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerID="8e571525f9002409f6b3780758b3d7c92d053b0aebe49c5d42dffaa86ea7420d" exitCode=0
Oct 30 19:33:08.520015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.519726    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" event={"ID":"f37fc0e1-0948-462d-83c2-64803cd0151f","Type":"ContainerDied","Data":"8e571525f9002409f6b3780758b3d7c92d053b0aebe49c5d42dffaa86ea7420d"}
Oct 30 19:33:08.520015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.519750    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf" event={"ID":"f37fc0e1-0948-462d-83c2-64803cd0151f","Type":"ContainerDied","Data":"1629dec1bd02a3d2cf64ff7d93f1d2a1a1f689aec87e54680d6f328f103f8a47"}
Oct 30 19:33:08.520015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.519760    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1629dec1bd02a3d2cf64ff7d93f1d2a1a1f689aec87e54680d6f328f103f8a47"
Oct 30 19:33:08.569806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.569776    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:33:08.572297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.572265    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:33:08.595451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595423    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.595593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595459    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.595593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595480    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\" (UniqueName: \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.595593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595536    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.595593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595576    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595610    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595662    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595700    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\" (UniqueName: \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595727    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\" (UniqueName: \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595749    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.595817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595793    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\") pod \"f37fc0e1-0948-462d-83c2-64803cd0151f\" (UID: \"f37fc0e1-0948-462d-83c2-64803cd0151f\") "
Oct 30 19:33:08.596110 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.595840    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\") pod \"ae35af12-8cd7-47bb-8031-0bfd1412b590\" (UID: \"ae35af12-8cd7-47bb-8031-0bfd1412b590\") "
Oct 30 19:33:08.596176 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.596149    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.596564 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.596531    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.596813 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.596789    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.596962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.596933    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.597397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.597370    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.598145 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.598119    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.601135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.601070    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "kube-api-access-9csmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:08.601332 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.601301    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.601648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.601617    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "kube-api-access-7qpph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:08.601776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.601733    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs" pod "ae35af12-8cd7-47bb-8031-0bfd1412b590" (UID: "ae35af12-8cd7-47bb-8031-0bfd1412b590"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:08.602169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.602147    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:08.602786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.602748    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs" pod "f37fc0e1-0948-462d-83c2-64803cd0151f" (UID: "f37fc0e1-0948-462d-83c2-64803cd0151f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:08.698073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698040    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698073    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698087    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698101    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698115    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f37fc0e1-0948-462d-83c2-64803cd0151f-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698127    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae35af12-8cd7-47bb-8031-0bfd1412b590-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698144    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698158    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae35af12-8cd7-47bb-8031-0bfd1412b590-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698172    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698185    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9csmz\" (UniqueName: \"kubernetes.io/projected/f37fc0e1-0948-462d-83c2-64803cd0151f-kube-api-access-9csmz\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698219    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qpph\" (UniqueName: \"kubernetes.io/projected/ae35af12-8cd7-47bb-8031-0bfd1412b590-kube-api-access-7qpph\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:08.698295 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:08.698232    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f37fc0e1-0948-462d-83c2-64803cd0151f-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:09.528803 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.528776    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"
Oct 30 19:33:09.529212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.528802    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"
Oct 30 19:33:09.529212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.528804    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j" event={"ID":"ae35af12-8cd7-47bb-8031-0bfd1412b590","Type":"ContainerDied","Data":"5a44cded5415d2b45b4c33c1903f7673f70fe49e0b6a981c4482aca820f77dd3"}
Oct 30 19:33:09.529212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.528851    5033 scope.go:122] "RemoveContainer" containerID="e45a7e9235e9b321a036b92047806c06b74a059fd5423fbcccc39d7c675fd8d6"
Oct 30 19:33:09.550373 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.550183    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:33:09.555023 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.554983    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/distinct-2-w1-6cf459-kserve-64ffb77c6d-xd5nf"]
Oct 30 19:33:09.566421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.566400    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:33:09.573650 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:09.573630    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-6cf459/premium-w1-6cf459-kserve-6f6bf66644-8cr4j"]
Oct 30 19:33:11.377698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:11.376443    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" path="/var/lib/kubelet/pods/ae35af12-8cd7-47bb-8031-0bfd1412b590/volumes"
Oct 30 19:33:11.377698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:11.377168    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" path="/var/lib/kubelet/pods/f37fc0e1-0948-462d-83c2-64803cd0151f/volumes"
Oct 30 19:33:12.145698 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.141884    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:33:12.767701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.767374    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:33:12.767701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.767628    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy" containerID="cri-o://6dba2239835baaf4f4559185ae7743e34639eb2dd9b1e3b1565b6d3fa5513552" gracePeriod=30
Oct 30 19:33:12.835964 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.835932    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:33:12.836467 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.836411    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy" containerID="cri-o://5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059" gracePeriod=30
Oct 30 19:33:12.884708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.881731    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:33:12.884708 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.881989    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" containerID="cri-o://6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451" gracePeriod=30
Oct 30 19:33:12.915710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.913782    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:33:12.915710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:12.914016    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy" containerID="cri-o://3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970" gracePeriod=30
Oct 30 19:33:13.736124 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.736092    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:33:13.798470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.787525    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\") pod \"6c4f240a-9663-46e3-8114-73c9b4ae0501\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") "
Oct 30 19:33:13.798470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.787670    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\" (UniqueName: \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\") pod \"6c4f240a-9663-46e3-8114-73c9b4ae0501\" (UID: \"6c4f240a-9663-46e3-8114-73c9b4ae0501\") "
Oct 30 19:33:13.798470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.798401    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume" pod "6c4f240a-9663-46e3-8114-73c9b4ae0501" (UID: "6c4f240a-9663-46e3-8114-73c9b4ae0501"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:13.835804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.810429    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj" pod "6c4f240a-9663-46e3-8114-73c9b4ae0501" (UID: "6c4f240a-9663-46e3-8114-73c9b4ae0501"). InnerVolumeSpecName "kube-api-access-xm7cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:13.835804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.818451    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"]
Oct 30 19:33:13.835804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.834985    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-6cf459-5f4xj"]
Oct 30 19:33:13.856528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.856496    5033 generic.go:356] "Generic (PLEG): container finished" podID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerID="30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f" exitCode=0
Oct 30 19:33:13.880003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.856794    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"
Oct 30 19:33:13.880003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.856861    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" event={"ID":"6c4f240a-9663-46e3-8114-73c9b4ae0501","Type":"ContainerDied","Data":"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"}
Oct 30 19:33:13.880003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.856919    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh" event={"ID":"6c4f240a-9663-46e3-8114-73c9b4ae0501","Type":"ContainerDied","Data":"e73473e5e3090bbdc9a54dd8d60fc2f026b89a4c043683e22df33dc8d751755d"}
Oct 30 19:33:13.880003 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.856950    5033 scope.go:122] "RemoveContainer" containerID="30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"
Oct 30 19:33:13.897986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.897940    5033 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4f240a-9663-46e3-8114-73c9b4ae0501-plugins-config-volume\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:13.898195 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.898174    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm7cj\" (UniqueName: \"kubernetes.io/projected/6c4f240a-9663-46e3-8114-73c9b4ae0501-kube-api-access-xm7cj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:13.938177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.938153    5033 scope.go:122] "RemoveContainer" containerID="30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"
Oct 30 19:33:13.938860 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:13.938784    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f\": container with ID starting with 30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f not found: ID does not exist" containerID="30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"
Oct 30 19:33:13.938860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.938827    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f"} err="failed to get container status \"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f\": rpc error: code = NotFound desc = could not find container \"30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f\": container with ID starting with 30f51a1680eef74959458f92bc9ada19bf664000071171735b77a0d08af7ca1f not found: ID does not exist"
Oct 30 19:33:13.972347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.972290    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:33:13.976922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:13.976896    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-639c15-5b945455c9-7zlvh"]
Oct 30 19:33:14.349519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.349484    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"]
Oct 30 19:33:14.350335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350307    5033 cpu_manager.go:414] "containerMap: removing container" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main"
Oct 30 19:33:14.350335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350336    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350355    5033 cpu_manager.go:414] "containerMap: removing container" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350364    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350383    5033 cpu_manager.go:414] "containerMap: removing container" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350391    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350413    5033 cpu_manager.go:414] "containerMap: removing container" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350420    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350432    5033 cpu_manager.go:414] "containerMap: removing container" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350439    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350459    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350466    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350478    5033 cpu_manager.go:414] "containerMap: removing container" podUID="bdb58caf-9f9d-43b1-ac07-0f402058348e" containerName="revoke-keys"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350485    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdb58caf-9f9d-43b1-ac07-0f402058348e" containerName="revoke-keys"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350504    5033 cpu_manager.go:414] "containerMap: removing container" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing"
Oct 30 19:33:14.350518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350512    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350526    5033 cpu_manager.go:414] "containerMap: removing container" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350533    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350553    5033 cpu_manager.go:414] "containerMap: removing container" podUID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" containerName="maas-api"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350560    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" containerName="maas-api"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350571    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4aea349b-f447-472d-8e0a-a6e9ca9e96f6" containerName="revoke-keys"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350578    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aea349b-f447-472d-8e0a-a6e9ca9e96f6" containerName="revoke-keys"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350729    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="c01c0ecd-311a-4c1b-8d8a-af3e8debaef5" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350743    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdb58caf-9f9d-43b1-ac07-0f402058348e" containerName="revoke-keys"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350756    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" containerName="payload-pre-processing"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350766    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae35af12-8cd7-47bb-8031-0bfd1412b590" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350777    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="b84a3a0a-1edc-4e03-a4a5-1e7862b14bda" containerName="maas-api"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350786    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aea349b-f447-472d-8e0a-a6e9ca9e96f6" containerName="revoke-keys"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350798    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="aae483f3-d484-4f73-8ec4-dfde145ea837" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350805    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="a27fa030-73b4-4864-8aaf-29bc3451860d" containerName="istio-proxy"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350816    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="f37fc0e1-0948-462d-83c2-64803cd0151f" containerName="main"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350825    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fde99e3-88c5-44d9-bc8c-20173f17f42b" containerName="istio-proxy"
Oct 30 19:33:14.351007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.350834    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd61b096-84b3-4bb1-a3f1-dd1c36cf5ef9" containerName="istio-proxy"
Oct 30 19:33:14.356468 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.356444    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.365852 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.365825    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:14.371805 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.371784    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:33:14.372375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.372351    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"]
Oct 30 19:33:14.404152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.404118    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7lpw\" (UniqueName: \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.404353 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.404183    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.404353 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.404335    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.505466 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.505431    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.505630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.505513    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.505689 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.505661    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7lpw\" (UniqueName: \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.506228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.506202    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.507860 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.507839    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.515722 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.515700    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7lpw\" (UniqueName: \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\") pod \"maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.674863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.674791    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:14.688386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.688357    5033 patch_prober.go:29] interesting pod/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:14.688501 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.688412    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:14.961564 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:33:14.961507    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e08dfb0_94a2_42ca_827d_32275e24a173.slice/crio-729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b WatchSource:0}: Error finding container 729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b: Status 404 returned error can't find the container with id 729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b
Oct 30 19:33:14.962810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:14.962777    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"]
Oct 30 19:33:15.172951 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:15.172440    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16155a9f-35a5-4736-ac03-95a3a8743068" path="/var/lib/kubelet/pods/16155a9f-35a5-4736-ac03-95a3a8743068/volumes"
Oct 30 19:33:15.173144 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:15.173103    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c4f240a-9663-46e3-8114-73c9b4ae0501" path="/var/lib/kubelet/pods/6c4f240a-9663-46e3-8114-73c9b4ae0501/volumes"
Oct 30 19:33:15.895671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:15.895634    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps" event={"ID":"1e08dfb0-94a2-42ca-827d-32275e24a173","Type":"ContainerStarted","Data":"aea92ca2c692c35255b5db24a16b239a446d55d858bec387efbfc8810479f614"}
Oct 30 19:33:15.895857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:15.895725    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps" event={"ID":"1e08dfb0-94a2-42ca-827d-32275e24a173","Type":"ContainerStarted","Data":"729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b"}
Oct 30 19:33:15.919163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:15.919102    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps" podStartSLOduration=1.919082959 podStartE2EDuration="1.919082959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:15.915396047 +0000 UTC m=+860.885036883" watchObservedRunningTime="2026-10-30 19:33:15.919082959 +0000 UTC m=+860.888723797"
Oct 30 19:33:16.906857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:16.906811    5033 generic.go:356] "Generic (PLEG): container finished" podID="1e08dfb0-94a2-42ca-827d-32275e24a173" containerID="aea92ca2c692c35255b5db24a16b239a446d55d858bec387efbfc8810479f614" exitCode=0
Oct 30 19:33:16.907296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:16.906887    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps" event={"ID":"1e08dfb0-94a2-42ca-827d-32275e24a173","Type":"ContainerDied","Data":"aea92ca2c692c35255b5db24a16b239a446d55d858bec387efbfc8810479f614"}
Oct 30 19:33:17.564646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.564613    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:33:17.568726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.568706    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.583517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.583492    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:33:17.583517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.583505    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t7289\""
Oct 30 19:33:17.583716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.583543    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:33:17.583716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.583638    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:33:17.590602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.590580    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:33:17.656470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656442    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656588 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656563    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656626    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656770    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656875    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcgkn\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656951 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656937    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.656987 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.656975    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.757963 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.757930    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.757969    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.757993    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.758041    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.758071    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.758171    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.758370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.758263    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcgkn\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.759140 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.759081    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.761111 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.761082    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.761226 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.761111    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.763024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.763002    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.768398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.768374    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.769556 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.769531    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcgkn\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.776961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.776940    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-h5d5m\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.881861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.881839    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:17.918980 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.918956    5033 generic.go:356] "Generic (PLEG): container finished" podID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerID="6dba2239835baaf4f4559185ae7743e34639eb2dd9b1e3b1565b6d3fa5513552" exitCode=0
Oct 30 19:33:17.919294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:17.919033    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" event={"ID":"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d","Type":"ContainerDied","Data":"6dba2239835baaf4f4559185ae7743e34639eb2dd9b1e3b1565b6d3fa5513552"}
Oct 30 19:33:18.195340 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:33:18.195281    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc488364_b3a3_4802_adf0_f9211c1b3a6d.slice/crio-a6efce2af410904b24ebf79a1f73a9661ad4092ebb9d154f36779ec1f339aa3b WatchSource:0}: Error finding container a6efce2af410904b24ebf79a1f73a9661ad4092ebb9d154f36779ec1f339aa3b: Status 404 returned error can't find the container with id a6efce2af410904b24ebf79a1f73a9661ad4092ebb9d154f36779ec1f339aa3b
Oct 30 19:33:18.197598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.197350    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:33:18.199829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.199809    5033 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:33:18.260804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.260774    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:18.333393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.332404    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:33:18.336332 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.336308    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:33:18.339889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.339870    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:33:18.407536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.407429    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\") pod \"1e08dfb0-94a2-42ca-827d-32275e24a173\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") "
Oct 30 19:33:18.408025 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.407998    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.408246 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.408229    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.408441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.408426    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.408580 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.408563    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.408776 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.408758    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\" (UniqueName: \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\") pod \"1e08dfb0-94a2-42ca-827d-32275e24a173\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") "
Oct 30 19:33:18.408921 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.408906    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.409078 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409062    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\") pod \"1e08dfb0-94a2-42ca-827d-32275e24a173\" (UID: \"1e08dfb0-94a2-42ca-827d-32275e24a173\") "
Oct 30 19:33:18.409646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409628    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.409874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409570    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca" pod "1e08dfb0-94a2-42ca-827d-32275e24a173" (UID: "1e08dfb0-94a2-42ca-827d-32275e24a173"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409873    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409898    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.409936    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410008    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410056    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410099    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410150    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410185    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410241    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410243    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410284    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410318    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410385    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410416    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410452    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410520    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410598    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410639    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410690    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410716    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\") pod \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\" (UID: \"534601b5-8ae0-46c4-9acb-7e7c9f967cc4\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410762    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410828    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410861    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\") pod \"97fbfe14-3068-48aa-961e-3f38960ce9ab\" (UID: \"97fbfe14-3068-48aa-961e-3f38960ce9ab\") "
Oct 30 19:33:18.411013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410915    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\") pod \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\" (UID: \"ec69a62c-7143-4ef9-849e-15d6b5529ea2\") "
Oct 30 19:33:18.411801 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.411672    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.411801 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.411782    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.411801 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.411797    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.411904 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.411812    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.412118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.410638    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.412185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.412155    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca" pod "1e08dfb0-94a2-42ca-827d-32275e24a173" (UID: "1e08dfb0-94a2-42ca-827d-32275e24a173"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.412523    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.412556    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.413045    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.413070    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.413390    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.413783    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.413876    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.414084    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.414660 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.414242    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.415536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.414702    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.416280 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.416235    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:18.416890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.416867    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.418208 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.418169    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.418296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.418273    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "kube-api-access-vdvmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.418441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.418410    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:18.421113 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421078    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.421307 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421240    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "kube-api-access-gjdmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.421307 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421257    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.421307 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421280    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token" pod "97fbfe14-3068-48aa-961e-3f38960ce9ab" (UID: "97fbfe14-3068-48aa-961e-3f38960ce9ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.421861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421517    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "kube-api-access-j7sb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.421861 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421618    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy" pod "534601b5-8ae0-46c4-9acb-7e7c9f967cc4" (UID: "534601b5-8ae0-46c4-9acb-7e7c9f967cc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.422057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.421877    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:18.423151 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.423118    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw" pod "1e08dfb0-94a2-42ca-827d-32275e24a173" (UID: "1e08dfb0-94a2-42ca-827d-32275e24a173"). InnerVolumeSpecName "kube-api-access-q7lpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.423385 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.423362    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy" pod "ec69a62c-7143-4ef9-849e-15d6b5529ea2" (UID: "ec69a62c-7143-4ef9-849e-15d6b5529ea2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.513778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513744    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7sb4\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-kube-api-access-j7sb4\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513782    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513798    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513812    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7lpw\" (UniqueName: \"kubernetes.io/projected/1e08dfb0-94a2-42ca-827d-32275e24a173-kube-api-access-q7lpw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513827    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjdmr\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-kube-api-access-gjdmr\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513841    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513855    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513870    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513884    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513897    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513910    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513927    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fbfe14-3068-48aa-961e-3f38960ce9ab-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513941    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513954    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdvmc\" (UniqueName: \"kubernetes.io/projected/97fbfe14-3068-48aa-961e-3f38960ce9ab-kube-api-access-vdvmc\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513967    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.513982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513980    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.513993    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514006    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec69a62c-7143-4ef9-849e-15d6b5529ea2-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514021    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514035    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514050    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/534601b5-8ae0-46c4-9acb-7e7c9f967cc4-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514063    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514076    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514090    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fbfe14-3068-48aa-961e-3f38960ce9ab-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514103    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec69a62c-7143-4ef9-849e-15d6b5529ea2-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.514517 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.514117    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e08dfb0-94a2-42ca-827d-32275e24a173-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.561331 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.561300    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:33:18.716359 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716325    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716368    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716405    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716435    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716466    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716512 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716498    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716566    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716596    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716642    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\") pod \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\" (UID: \"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d\") "
Oct 30 19:33:18.716785 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716750    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.717008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716797    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.717008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716877    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:18.717008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.716932    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.717341 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.717270    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.717458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.717356    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.717458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.717377    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.717458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.717406    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.717458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.717420    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.718848 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.718824    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:18.719260 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.719237    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:18.719342 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.719242    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.719380 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.719341    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6" pod "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" (UID: "c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d"). InnerVolumeSpecName "kube-api-access-572p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:18.818337 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.818306    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.818337 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.818337    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.818496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.818352    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.818496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.818365    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-572p6\" (UniqueName: \"kubernetes.io/projected/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-kube-api-access-572p6\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.818496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.818379    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:18.929401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.929355    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" event={"ID":"bc488364-b3a3-4802-adf0-f9211c1b3a6d","Type":"ContainerStarted","Data":"a6efce2af410904b24ebf79a1f73a9661ad4092ebb9d154f36779ec1f339aa3b"}
Oct 30 19:33:18.931414 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.931379    5033 generic.go:356] "Generic (PLEG): container finished" podID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerID="5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059" exitCode=0
Oct 30 19:33:18.931546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.931444    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"
Oct 30 19:33:18.931546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.931461    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" event={"ID":"ec69a62c-7143-4ef9-849e-15d6b5529ea2","Type":"ContainerDied","Data":"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"}
Oct 30 19:33:18.931546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.931516    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf" event={"ID":"ec69a62c-7143-4ef9-849e-15d6b5529ea2","Type":"ContainerDied","Data":"dfd077751cb3f1b34617b82c2017cef06fee028b6611d1a3682815c6ceb69a17"}
Oct 30 19:33:18.931546 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.931546    5033 scope.go:122] "RemoveContainer" containerID="5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"
Oct 30 19:33:18.933349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.933320    5033 generic.go:356] "Generic (PLEG): container finished" podID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerID="6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451" exitCode=0
Oct 30 19:33:18.933478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.933372    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"
Oct 30 19:33:18.933478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.933399    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" event={"ID":"97fbfe14-3068-48aa-961e-3f38960ce9ab","Type":"ContainerDied","Data":"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"}
Oct 30 19:33:18.933478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.933453    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts" event={"ID":"97fbfe14-3068-48aa-961e-3f38960ce9ab","Type":"ContainerDied","Data":"72e0c1e58e1e9f9130d564d304e88136cc8464d34f4c25b0426e3241ba489666"}
Oct 30 19:33:18.949693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.946972    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps" event={"ID":"1e08dfb0-94a2-42ca-827d-32275e24a173","Type":"ContainerDied","Data":"729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b"}
Oct 30 19:33:18.949693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.947017    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="729bc949125e3ab16fcc0e96cb536d5163383c2d8c8079f3f5de632a5146d38b"
Oct 30 19:33:18.949693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.947079    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"
Oct 30 19:33:18.956789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.956765    5033 scope.go:122] "RemoveContainer" containerID="5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:18.958344    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059\": container with ID starting with 5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059 not found: ID does not exist" containerID="5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.958382    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059"} err="failed to get container status \"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059\": rpc error: code = NotFound desc = could not find container \"5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059\": container with ID starting with 5a801d5291063b4553dbc179995151c29121fb26cbc5bb2b8ea81d39b0a09059 not found: ID does not exist"
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.958405    5033 scope.go:122] "RemoveContainer" containerID="6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.959739    5033 generic.go:356] "Generic (PLEG): container finished" podID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerID="3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970" exitCode=0
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.959804    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" event={"ID":"534601b5-8ae0-46c4-9acb-7e7c9f967cc4","Type":"ContainerDied","Data":"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"}
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.959843    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv" event={"ID":"534601b5-8ae0-46c4-9acb-7e7c9f967cc4","Type":"ContainerDied","Data":"c18285cacdea3efb1168e0bd51df162909468c5a4b581365040216366b2b31ba"}
Oct 30 19:33:18.962128 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.959910    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"
Oct 30 19:33:18.974893 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.974811    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n" event={"ID":"c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d","Type":"ContainerDied","Data":"4f8579c97856acce0d3bdb1013f1069e43ed4708cc0b40eec92bb9dc5c3aca9b"}
Oct 30 19:33:18.975045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.974919    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"
Oct 30 19:33:18.986646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.984107    5033 scope.go:122] "RemoveContainer" containerID="6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"
Oct 30 19:33:18.986646 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:18.984710    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451\": container with ID starting with 6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451 not found: ID does not exist" containerID="6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"
Oct 30 19:33:18.986646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.984747    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451"} err="failed to get container status \"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451\": rpc error: code = NotFound desc = could not find container \"6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451\": container with ID starting with 6226c825fdf2d3eb105d29e905bcdde03c668d9ea7c71854d9f14ef772501451 not found: ID does not exist"
Oct 30 19:33:18.986646 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:18.984792    5033 scope.go:122] "RemoveContainer" containerID="3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"
Oct 30 19:33:19.007985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.007960    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:33:19.011478 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.011379    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-qj6zf"]
Oct 30 19:33:19.011583 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.011570    5033 scope.go:122] "RemoveContainer" containerID="3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"
Oct 30 19:33:19.011947 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:19.011925    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970\": container with ID starting with 3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970 not found: ID does not exist" containerID="3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"
Oct 30 19:33:19.012029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.011953    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970"} err="failed to get container status \"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970\": rpc error: code = NotFound desc = could not find container \"3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970\": container with ID starting with 3b91205d9c4562d33c9b830b8a86485de5f9e7275a5f49f592b28813d709d970 not found: ID does not exist"
Oct 30 19:33:19.012029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.011972    5033 scope.go:122] "RemoveContainer" containerID="6dba2239835baaf4f4559185ae7743e34639eb2dd9b1e3b1565b6d3fa5513552"
Oct 30 19:33:19.031602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.031573    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:33:19.035204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.035184    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9psts"]
Oct 30 19:33:19.065713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.065668    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:33:19.069491 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.069471    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-9v22n"]
Oct 30 19:33:19.087874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.087853    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:33:19.093201 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.093178    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-639c15-openshift-default-58d59f577f-27hzv"]
Oct 30 19:33:19.173187 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.173155    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" path="/var/lib/kubelet/pods/534601b5-8ae0-46c4-9acb-7e7c9f967cc4/volumes"
Oct 30 19:33:19.173648 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.173635    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" path="/var/lib/kubelet/pods/97fbfe14-3068-48aa-961e-3f38960ce9ab/volumes"
Oct 30 19:33:19.174117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.174102    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" path="/var/lib/kubelet/pods/c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d/volumes"
Oct 30 19:33:19.174889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.174873    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" path="/var/lib/kubelet/pods/ec69a62c-7143-4ef9-849e-15d6b5529ea2/volumes"
Oct 30 19:33:19.175188 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:19.175175    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:33:21.653364 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.653318    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:33:21.654068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.654036    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" containerID="cri-o://87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f" gracePeriod=30
Oct 30 19:33:21.757514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.757458    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:33:21.758057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.757707    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy" containerID="cri-o://e76f880b7609eb02456f1477d195680e9d5d4abf149ab4ffec9b7e2c8f94cbf1" gracePeriod=30
Oct 30 19:33:21.799238 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.799204    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:33:21.812472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:21.812176    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy" containerID="cri-o://3aba93ac98bcc1a29bfa64e6e058cf6e90ae59bfb78f540b23fff95851d14954" gracePeriod=30
Oct 30 19:33:22.101431 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:22.101386    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:33:22.101560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:22.101478    5033 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:33:23.036753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.036712    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" event={"ID":"bc488364-b3a3-4802-adf0-f9211c1b3a6d","Type":"ContainerStarted","Data":"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"}
Oct 30 19:33:23.037216 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.036993    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:23.038988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.038963    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:33:23.069742 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.069696    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" podStartSLOduration=2.170014536 podStartE2EDuration="6.069669942s" totalImagesPullingTime="3.899655406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:23.066521474 +0000 UTC m=+868.036162312" watchObservedRunningTime="2026-10-30 19:33:23.069669942 +0000 UTC m=+868.039310776"
Oct 30 19:33:23.967005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.966975    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:33:23.967458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:23.967424    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" containerID="cri-o://4ac8f0b61e5f92733851ca600bf100b7df96abe3428e260a560f93551b255593" gracePeriod=30
Oct 30 19:33:24.006764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.006730    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:33:24.007044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.007015    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" containerID="cri-o://0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369" gracePeriod=30
Oct 30 19:33:24.012837 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.012805    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:33:24.013086 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.013041    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy" containerID="cri-o://d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276" gracePeriod=30
Oct 30 19:33:24.311754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.311649    5033 patch_prober.go:29] interesting pod/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:24.311754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.311735    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:24.804775 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.804736    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"]
Oct 30 19:33:24.805535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805515    5033 cpu_manager.go:414] "containerMap: removing container" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy"
Oct 30 19:33:24.805599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805539    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy"
Oct 30 19:33:24.805599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805559    5033 cpu_manager.go:414] "containerMap: removing container" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy"
Oct 30 19:33:24.805599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805566    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy"
Oct 30 19:33:24.805599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805589    5033 cpu_manager.go:414] "containerMap: removing container" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy"
Oct 30 19:33:24.805599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805597    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy"
Oct 30 19:33:24.805792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805616    5033 cpu_manager.go:414] "containerMap: removing container" podUID="1e08dfb0-94a2-42ca-827d-32275e24a173" containerName="revoke-keys"
Oct 30 19:33:24.805792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805625    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e08dfb0-94a2-42ca-827d-32275e24a173" containerName="revoke-keys"
Oct 30 19:33:24.805792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805668    5033 cpu_manager.go:414] "containerMap: removing container" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy"
Oct 30 19:33:24.805792 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805692    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy"
Oct 30 19:33:24.805912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805806    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="97fbfe14-3068-48aa-961e-3f38960ce9ab" containerName="istio-proxy"
Oct 30 19:33:24.805912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805815    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e08dfb0-94a2-42ca-827d-32275e24a173" containerName="revoke-keys"
Oct 30 19:33:24.805912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805825    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec69a62c-7143-4ef9-849e-15d6b5529ea2" containerName="istio-proxy"
Oct 30 19:33:24.805912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805833    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="534601b5-8ae0-46c4-9acb-7e7c9f967cc4" containerName="istio-proxy"
Oct 30 19:33:24.805912 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.805839    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="c584e0cc-8bb3-4ce2-8b2a-5beaa6c8741d" containerName="istio-proxy"
Oct 30 19:33:24.809739 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.809713    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.819311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.819284    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"]
Oct 30 19:33:24.835402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.834239    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.835402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.834315    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw4t8\" (UniqueName: \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.835402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.834365    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.855169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.854834    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:24.855169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.854900    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:24.900152 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.900113    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:24.900320 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.900202    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:33:24.935136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.935102    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.935326 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.935307    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.935401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.935375    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw4t8\" (UniqueName: \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.935882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.935859    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.937763 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.937740    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:24.947009 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:24.946985    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw4t8\" (UniqueName: \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\") pod \"maas-api-revoke-sub-b8151a70-gm9xz\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:25.179817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:25.179732    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:25.465855 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:25.465824    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"]
Oct 30 19:33:25.466211 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:33:25.465867    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9e9034_bc0f_4b62_bec2_f02157fcbf78.slice/crio-5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166 WatchSource:0}: Error finding container 5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166: Status 404 returned error can't find the container with id 5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166
Oct 30 19:33:26.062189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.062097    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz" event={"ID":"bd9e9034-bc0f-4b62-bec2-f02157fcbf78","Type":"ContainerStarted","Data":"109167b65dd853abab096c51dadfcb49b758040764b662d65d5bcaffa95c871f"}
Oct 30 19:33:26.062189 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.062158    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz" event={"ID":"bd9e9034-bc0f-4b62-bec2-f02157fcbf78","Type":"ContainerStarted","Data":"5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166"}
Oct 30 19:33:26.080982 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.080937    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz" podStartSLOduration=2.080925874 podStartE2EDuration="2.080925874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:26.077957021 +0000 UTC m=+871.047597855" watchObservedRunningTime="2026-10-30 19:33:26.080925874 +0000 UTC m=+871.050566709"
Oct 30 19:33:26.590632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.590598    5033 patch_prober.go:29] interesting pod/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:26.591049 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.590658    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:26.937858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.937834    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.950720    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.950833    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.950871    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.950928    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.950985    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.951087    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.951188    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.951250    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.951280    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\") pod \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\" (UID: \"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032\") "
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.952104    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.952364    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.952720    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:26.952863 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.952786    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:26.953658 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.953623    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:26.954898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.954851    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:26.955728 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.955663    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "kube-api-access-9rj26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:26.962817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.956457    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:26.962817 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:26.957603    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy" pod "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" (UID: "d34d5d61-7e02-4fcf-9fc9-178f8bc2f032"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053631    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053666    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053696    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053732    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053747    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053759    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053772    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053785    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rj26\" (UniqueName: \"kubernetes.io/projected/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-kube-api-access-9rj26\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.068608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.053798    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.085988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.082803    5033 generic.go:356] "Generic (PLEG): container finished" podID="bd9e9034-bc0f-4b62-bec2-f02157fcbf78" containerID="109167b65dd853abab096c51dadfcb49b758040764b662d65d5bcaffa95c871f" exitCode=0
Oct 30 19:33:27.085988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.082988    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz" event={"ID":"bd9e9034-bc0f-4b62-bec2-f02157fcbf78","Type":"ContainerDied","Data":"109167b65dd853abab096c51dadfcb49b758040764b662d65d5bcaffa95c871f"}
Oct 30 19:33:27.085988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.085570    5033 generic.go:356] "Generic (PLEG): container finished" podID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerID="3aba93ac98bcc1a29bfa64e6e058cf6e90ae59bfb78f540b23fff95851d14954" exitCode=0
Oct 30 19:33:27.085988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.085638    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" event={"ID":"db6c61fe-0de4-4c23-8753-d1a0b016a9a4","Type":"ContainerDied","Data":"3aba93ac98bcc1a29bfa64e6e058cf6e90ae59bfb78f540b23fff95851d14954"}
Oct 30 19:33:27.087435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.087409    5033 generic.go:356] "Generic (PLEG): container finished" podID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerID="87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f" exitCode=0
Oct 30 19:33:27.087552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.087461    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"
Oct 30 19:33:27.087552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.087504    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" event={"ID":"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032","Type":"ContainerDied","Data":"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"}
Oct 30 19:33:27.087552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.087541    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk" event={"ID":"d34d5d61-7e02-4fcf-9fc9-178f8bc2f032","Type":"ContainerDied","Data":"b50757eccc29191718fcb6210d671ee29e94bf31c704622c559fb3ad4232c9a9"}
Oct 30 19:33:27.087715 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.087568    5033 scope.go:122] "RemoveContainer" containerID="87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"
Oct 30 19:33:27.090011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.089896    5033 generic.go:356] "Generic (PLEG): container finished" podID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerID="e76f880b7609eb02456f1477d195680e9d5d4abf149ab4ffec9b7e2c8f94cbf1" exitCode=0
Oct 30 19:33:27.090011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.089953    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" event={"ID":"36c4f614-8b00-4946-8f5e-2df9d67b6d39","Type":"ContainerDied","Data":"e76f880b7609eb02456f1477d195680e9d5d4abf149ab4ffec9b7e2c8f94cbf1"}
Oct 30 19:33:27.112730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.112713    5033 scope.go:122] "RemoveContainer" containerID="87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"
Oct 30 19:33:27.113019 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:27.112998    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f\": container with ID starting with 87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f not found: ID does not exist" containerID="87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"
Oct 30 19:33:27.113090 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.113028    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f"} err="failed to get container status \"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f\": rpc error: code = NotFound desc = could not find container \"87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f\": container with ID starting with 87f736fc468467e0245f0b82acab31ef678804a39f0719a555bb9cb26f39b03f not found: ID does not exist"
Oct 30 19:33:27.114885 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.114862    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:33:27.126922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.126905    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:33:27.155059 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155030    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155182 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155089    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155182 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155129    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155182 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155170    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155203    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155242    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155271    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155314    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155342    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155397    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155415    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155445    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155491    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155525    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155554    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155590    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.155632 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155616    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\") pod \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\" (UID: \"db6c61fe-0de4-4c23-8753-d1a0b016a9a4\") "
Oct 30 19:33:27.156067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.155638    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\") pod \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\" (UID: \"36c4f614-8b00-4946-8f5e-2df9d67b6d39\") "
Oct 30 19:33:27.156806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.156572    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:27.156806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.156609    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:27.156947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.156865    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157065    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157121    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157250    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157305    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157807 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157777    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.157997 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.157974    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.158073 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.158000    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.165888 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.165834    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.177134 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177096    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:27.177236 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177154    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "kube-api-access-qfs97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:27.177236 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177161    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "kube-api-access-qksrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:27.177384 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177273    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:27.177568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177540    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:27.177668 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.177646    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo" pod "36c4f614-8b00-4946-8f5e-2df9d67b6d39" (UID: "36c4f614-8b00-4946-8f5e-2df9d67b6d39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:27.180088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.180025    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy" pod "db6c61fe-0de4-4c23-8753-d1a0b016a9a4" (UID: "db6c61fe-0de4-4c23-8753-d1a0b016a9a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:27.184916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.184886    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:33:27.185022 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.184928    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-c6tqk"]
Oct 30 19:33:27.257375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257348    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257374    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257383    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257392    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257401    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qksrb\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-kube-api-access-qksrb\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257410    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257420    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfs97\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-kube-api-access-qfs97\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257433    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257446    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257459    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257473    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257486    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257503    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257517    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257529    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257542    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257551    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c4f614-8b00-4946-8f5e-2df9d67b6d39-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:27.257961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:27.257560    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6c61fe-0de4-4c23-8753-d1a0b016a9a4-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:28.100245 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.100209    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"
Oct 30 19:33:28.100746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.100217    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr" event={"ID":"36c4f614-8b00-4946-8f5e-2df9d67b6d39","Type":"ContainerDied","Data":"a6a0bc2f467a5da2f56e18e2330a16acacb77bc678ed2c90a9dbe22ca85adca5"}
Oct 30 19:33:28.100746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.100363    5033 scope.go:122] "RemoveContainer" containerID="e76f880b7609eb02456f1477d195680e9d5d4abf149ab4ffec9b7e2c8f94cbf1"
Oct 30 19:33:28.102012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.101948    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"
Oct 30 19:33:28.102012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.101990    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767" event={"ID":"db6c61fe-0de4-4c23-8753-d1a0b016a9a4","Type":"ContainerDied","Data":"52f48653646f1aded62a79d662fed884d18518837e07e8532b5b21c12419a822"}
Oct 30 19:33:28.112480 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.112453    5033 scope.go:122] "RemoveContainer" containerID="3aba93ac98bcc1a29bfa64e6e058cf6e90ae59bfb78f540b23fff95851d14954"
Oct 30 19:33:28.125290 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.125270    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:33:28.131136 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.131112    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-ch767"]
Oct 30 19:33:28.153797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.153776    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:33:28.175500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.175471    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f7f075-openshift-default-7f77b5b897-mtrgr"]
Oct 30 19:33:28.252330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.252305    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:28.277251 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.277220    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\" (UniqueName: \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\") pod \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") "
Oct 30 19:33:28.277390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.277297    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\") pod \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") "
Oct 30 19:33:28.277448 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.277414    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\") pod \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\" (UID: \"bd9e9034-bc0f-4b62-bec2-f02157fcbf78\") "
Oct 30 19:33:28.278938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.278913    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca" pod "bd9e9034-bc0f-4b62-bec2-f02157fcbf78" (UID: "bd9e9034-bc0f-4b62-bec2-f02157fcbf78"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:28.280209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.280185    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca" pod "bd9e9034-bc0f-4b62-bec2-f02157fcbf78" (UID: "bd9e9034-bc0f-4b62-bec2-f02157fcbf78"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:28.280521 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.280504    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8" pod "bd9e9034-bc0f-4b62-bec2-f02157fcbf78" (UID: "bd9e9034-bc0f-4b62-bec2-f02157fcbf78"). InnerVolumeSpecName "kube-api-access-pw4t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:28.378661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.378630    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:28.378661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.378661    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw4t8\" (UniqueName: \"kubernetes.io/projected/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-kube-api-access-pw4t8\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:28.378869 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:28.378708    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9e9034-bc0f-4b62-bec2-f02157fcbf78-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.111311 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.111277    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"
Oct 30 19:33:29.111705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.111307    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz" event={"ID":"bd9e9034-bc0f-4b62-bec2-f02157fcbf78","Type":"ContainerDied","Data":"5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166"}
Oct 30 19:33:29.111705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.111360    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d68000e2cd650a471dffd0717ef775716c1cf48abd4472b6ab2c285ae1da166"
Oct 30 19:33:29.114518 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.114485    5033 generic.go:356] "Generic (PLEG): container finished" podID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerID="4ac8f0b61e5f92733851ca600bf100b7df96abe3428e260a560f93551b255593" exitCode=0
Oct 30 19:33:29.114625 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.114545    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" event={"ID":"9198743c-05d3-413a-a0b5-31ce2bf40c8a","Type":"ContainerDied","Data":"4ac8f0b61e5f92733851ca600bf100b7df96abe3428e260a560f93551b255593"}
Oct 30 19:33:29.175581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.175536    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" path="/var/lib/kubelet/pods/36c4f614-8b00-4946-8f5e-2df9d67b6d39/volumes"
Oct 30 19:33:29.176241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.176216    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" path="/var/lib/kubelet/pods/d34d5d61-7e02-4fcf-9fc9-178f8bc2f032/volumes"
Oct 30 19:33:29.176853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.176836    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" path="/var/lib/kubelet/pods/db6c61fe-0de4-4c23-8753-d1a0b016a9a4/volumes"
Oct 30 19:33:29.308563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.308535    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:33:29.342322 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.342062    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:33:29.389746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389709    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.389930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389774    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.389930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389807    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.389930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389841    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.389930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389879    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.389930 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389915    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.389949    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390013    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390041    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390085    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390130    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390165    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390209    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390244    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390279    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390317    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\") pod \"3e029a6d-6838-4509-a872-fe1dbb27b219\" (UID: \"3e029a6d-6838-4509-a872-fe1dbb27b219\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390363    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.390444 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.390422    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\") pod \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\" (UID: \"359196ec-6ea5-4c10-ac93-2cb03f88a4ee\") "
Oct 30 19:33:29.394383 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.394341    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.395131 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.395097    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.395299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.395144    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:29.395589 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.395565    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.396239 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.396191    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:29.396668 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.396646    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.398802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.398696    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.398906 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.398862    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.398976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.398939    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.399165 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.399146    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.399889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.399830    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.399985 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.399927    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.400961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.400910    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:29.401257 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.401201    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.401257 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.401239    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5" pod "359196ec-6ea5-4c10-ac93-2cb03f88a4ee" (UID: "359196ec-6ea5-4c10-ac93-2cb03f88a4ee"). InnerVolumeSpecName "kube-api-access-m2fp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.402041 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.401988    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:29.404581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.404552    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "kube-api-access-jfh9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.406769 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.406744    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs" pod "3e029a6d-6838-4509-a872-fe1dbb27b219" (UID: "3e029a6d-6838-4509-a872-fe1dbb27b219"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.492543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492508    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492539    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492548    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492557    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492565    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492573    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492581    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492589    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492597    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492604    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492612    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492643    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2fp5\" (UniqueName: \"kubernetes.io/projected/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-kube-api-access-m2fp5\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492652    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e029a6d-6838-4509-a872-fe1dbb27b219-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492661    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e029a6d-6838-4509-a872-fe1dbb27b219-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492670    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfh9d\" (UniqueName: \"kubernetes.io/projected/3e029a6d-6838-4509-a872-fe1dbb27b219-kube-api-access-jfh9d\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492713    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492726    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/359196ec-6ea5-4c10-ac93-2cb03f88a4ee-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.492774 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.492739    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e029a6d-6838-4509-a872-fe1dbb27b219-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.675874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.675845    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:33:29.694336 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694309    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694358    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694420    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694460    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694630 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694579    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694639    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694712    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694583    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.694851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694753    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.694851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694799    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.694851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694834    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\") pod \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\" (UID: \"9198743c-05d3-413a-a0b5-31ce2bf40c8a\") "
Oct 30 19:33:29.695088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.694800    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.695088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695062    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:29.695262 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695242    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.695318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695269    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.695318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695285    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.695318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695298    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.695469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.695371    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.697834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.697785    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:29.697973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.697842    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.698193 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.698174    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:29.698445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.698420    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw" pod "9198743c-05d3-413a-a0b5-31ce2bf40c8a" (UID: "9198743c-05d3-413a-a0b5-31ce2bf40c8a"). InnerVolumeSpecName "kube-api-access-gvxmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:29.796430 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.796388    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.796430 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.796427    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.796695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.796444    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvxmw\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-kube-api-access-gvxmw\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.796695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.796461    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:29.796695 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:29.796475    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9198743c-05d3-413a-a0b5-31ce2bf40c8a-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:30.151937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.151554    5033 generic.go:356] "Generic (PLEG): container finished" podID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerID="d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276" exitCode=0
Oct 30 19:33:30.151937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.151724    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerDied","Data":"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"}
Oct 30 19:33:30.151937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.151813    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg" event={"ID":"3e029a6d-6838-4509-a872-fe1dbb27b219","Type":"ContainerDied","Data":"cc2bf7ece57f9b13a5d479d2caf5df9d112c09aed2254110d2c9365884619ad9"}
Oct 30 19:33:30.151937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.151821    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"
Oct 30 19:33:30.151937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.151844    5033 scope.go:122] "RemoveContainer" containerID="d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"
Oct 30 19:33:30.156350 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.156122    5033 generic.go:356] "Generic (PLEG): container finished" podID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerID="0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369" exitCode=0
Oct 30 19:33:30.156350 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.156289    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" event={"ID":"359196ec-6ea5-4c10-ac93-2cb03f88a4ee","Type":"ContainerDied","Data":"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"}
Oct 30 19:33:30.156859 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.156432    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7" event={"ID":"359196ec-6ea5-4c10-ac93-2cb03f88a4ee","Type":"ContainerDied","Data":"9d71468052e180a0b85299328dbb6e949512cda3828cee24bab97d24f06ec6ff"}
Oct 30 19:33:30.156859 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.156630    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"
Oct 30 19:33:30.166272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.165076    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q" event={"ID":"9198743c-05d3-413a-a0b5-31ce2bf40c8a","Type":"ContainerDied","Data":"c01ea3123d0d0fa2dffebc19adf5bcc0020a60c3c3d49e9a4eacb5f44655449a"}
Oct 30 19:33:30.166272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.165158    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"
Oct 30 19:33:30.178185 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.175345    5033 scope.go:122] "RemoveContainer" containerID="462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"
Oct 30 19:33:30.201747 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.200057    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:33:30.205916 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.205895    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-x2zrg"]
Oct 30 19:33:30.206602 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.206580    5033 scope.go:122] "RemoveContainer" containerID="d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:30.207239    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276\": container with ID starting with d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276 not found: ID does not exist" containerID="d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.207271    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276"} err="failed to get container status \"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276\": rpc error: code = NotFound desc = could not find container \"d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276\": container with ID starting with d8b7a7b80caf5211b06e6d1c9eaaae6df0f0709ba0513de88aced9b274acb276 not found: ID does not exist"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.207288    5033 scope.go:122] "RemoveContainer" containerID="462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:30.207615    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3\": container with ID starting with 462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3 not found: ID does not exist" containerID="462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.207650    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3"} err="failed to get container status \"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3\": rpc error: code = NotFound desc = could not find container \"462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3\": container with ID starting with 462b96a25fe8ebd12d2bb935787a9aafcbc4d40d0b251b6afbda2b67f31e41c3 not found: ID does not exist"
Oct 30 19:33:30.209377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.207694    5033 scope.go:122] "RemoveContainer" containerID="0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"
Oct 30 19:33:30.220482 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.220449    5033 scope.go:122] "RemoveContainer" containerID="0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"
Oct 30 19:33:30.220802 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:30.220776    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369\": container with ID starting with 0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369 not found: ID does not exist" containerID="0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"
Oct 30 19:33:30.220892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.220811    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369"} err="failed to get container status \"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369\": rpc error: code = NotFound desc = could not find container \"0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369\": container with ID starting with 0de1ffd54664d5b32a61eb25e8b04eeca23b645de6b0e879dfa8f8f7157c8369 not found: ID does not exist"
Oct 30 19:33:30.220892 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.220835    5033 scope.go:122] "RemoveContainer" containerID="4ac8f0b61e5f92733851ca600bf100b7df96abe3428e260a560f93551b255593"
Oct 30 19:33:30.220988 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.220937    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:33:30.228375 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.228349    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-762j7"]
Oct 30 19:33:30.240605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.240586    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:33:30.271814 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:30.271698    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-6cf459-openshift-default-5cb8bc465c-8x24q"]
Oct 30 19:33:31.171173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:31.171139    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" path="/var/lib/kubelet/pods/359196ec-6ea5-4c10-ac93-2cb03f88a4ee/volumes"
Oct 30 19:33:31.171866 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:31.171846    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" path="/var/lib/kubelet/pods/3e029a6d-6838-4509-a872-fe1dbb27b219/volumes"
Oct 30 19:33:31.172519 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:31.172453    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" path="/var/lib/kubelet/pods/9198743c-05d3-413a-a0b5-31ce2bf40c8a/volumes"
Oct 30 19:33:34.198171 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.198130    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main" containerID="cri-o://928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f" gracePeriod=45
Oct 30 19:33:34.465184 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.465159    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:33:34.553975 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.553940    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\" (UniqueName: \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.554315 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.554289    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\" (UniqueName: \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.554494 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.554477    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.554634 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.554619    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.554877 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.554857    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.555052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.555035    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\") pod \"54432de9-313b-4824-b1f3-be4ca88764ef\" (UID: \"54432de9-313b-4824-b1f3-be4ca88764ef\") "
Oct 30 19:33:34.557503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.557476    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "kube-api-access-wn9p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:34.557762 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.557736    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:34.557922 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.557803    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:34.558008 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.557972    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:34.558118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.558095    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:34.558461 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.558438    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm" pod "54432de9-313b-4824-b1f3-be4ca88764ef" (UID: "54432de9-313b-4824-b1f3-be4ca88764ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:34.656968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656933    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.656968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656961    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.656968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656972    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn9p4\" (UniqueName: \"kubernetes.io/projected/54432de9-313b-4824-b1f3-be4ca88764ef-kube-api-access-wn9p4\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.657255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656982    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54432de9-313b-4824-b1f3-be4ca88764ef-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.657255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656990    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.657255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:34.656999    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54432de9-313b-4824-b1f3-be4ca88764ef-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:34.988455 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:34.988416    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:33:35.207389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.207157    5033 generic.go:356] "Generic (PLEG): container finished" podID="54432de9-313b-4824-b1f3-be4ca88764ef" containerID="928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f" exitCode=0
Oct 30 19:33:35.207389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.207207    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" event={"ID":"54432de9-313b-4824-b1f3-be4ca88764ef","Type":"ContainerDied","Data":"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"}
Oct 30 19:33:35.207389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.207266    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq" event={"ID":"54432de9-313b-4824-b1f3-be4ca88764ef","Type":"ContainerDied","Data":"47188d42b22192816aa1da91b5e27b7fc85bc567caeb79fda2e4c10e09d2eb3b"}
Oct 30 19:33:35.207389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.207289    5033 scope.go:122] "RemoveContainer" containerID="928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"
Oct 30 19:33:35.207389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.207218    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"
Oct 30 19:33:35.219873 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.219856    5033 scope.go:122] "RemoveContainer" containerID="928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"
Oct 30 19:33:35.220155 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:35.220130    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f\": container with ID starting with 928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f not found: ID does not exist" containerID="928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"
Oct 30 19:33:35.220239 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.220165    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f"} err="failed to get container status \"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f\": rpc error: code = NotFound desc = could not find container \"928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f\": container with ID starting with 928a70604db868dbf2cc08ec2a2facd81da4dfb3c7a4b9038deabf735271746f not found: ID does not exist"
Oct 30 19:33:35.236207 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.236182    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:33:35.242761 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:35.242708    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3bc085/simulator-w4-3bc085-kserve-5c77d9798d-qq7nq"]
Oct 30 19:33:37.170010 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:37.169976    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" path="/var/lib/kubelet/pods/54432de9-313b-4824-b1f3-be4ca88764ef/volumes"
Oct 30 19:33:40.164051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:40.164015    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"]
Oct 30 19:33:40.170997 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:40.170966    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3bc085-t4zps"]
Oct 30 19:33:41.168685 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:41.168651    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e08dfb0-94a2-42ca-827d-32275e24a173" path="/var/lib/kubelet/pods/1e08dfb0-94a2-42ca-827d-32275e24a173/volumes"
Oct 30 19:33:47.268968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.268934    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:33:47.269370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.269129    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" containerID="cri-o://f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af" gracePeriod=30
Oct 30 19:33:47.344379 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.343721    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:33:47.344379 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.343969    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy" containerID="cri-o://f6ca135896633d11d6fe96ad81313aa4eace07353e14ad42a60cffbf568672e5" gracePeriod=30
Oct 30 19:33:47.350289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.350229    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:33:47.350572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:47.350462    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" containerID="cri-o://5e6959dede6f7f644bc71661398d2243aa865a9e115866bb2eed1fc0e1f16338" gracePeriod=30
Oct 30 19:33:49.975051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:49.975000    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:33:49.975558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:49.975235    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-78455b774c-skgjw" podUID="7827f4c0-6c4d-48e4-959a-47188b2693ab" containerName="manager" containerID="cri-o://6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9" gracePeriod=30
Oct 30 19:33:50.083565 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.083532    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:50.083700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.083595    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:50.253598 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.253572    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:33:50.329538 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.329488    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\" (UniqueName: \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\") pod \"7827f4c0-6c4d-48e4-959a-47188b2693ab\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") "
Oct 30 19:33:50.329740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.329577    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\") pod \"7827f4c0-6c4d-48e4-959a-47188b2693ab\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") "
Oct 30 19:33:50.329740 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.329643    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\") pod \"7827f4c0-6c4d-48e4-959a-47188b2693ab\" (UID: \"7827f4c0-6c4d-48e4-959a-47188b2693ab\") "
Oct 30 19:33:50.332305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.332238    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert" pod "7827f4c0-6c4d-48e4-959a-47188b2693ab" (UID: "7827f4c0-6c4d-48e4-959a-47188b2693ab"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:50.332419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.332352    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs" pod "7827f4c0-6c4d-48e4-959a-47188b2693ab" (UID: "7827f4c0-6c4d-48e4-959a-47188b2693ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:50.332482 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.332460    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h" pod "7827f4c0-6c4d-48e4-959a-47188b2693ab" (UID: "7827f4c0-6c4d-48e4-959a-47188b2693ab"). InnerVolumeSpecName "kube-api-access-pws7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:50.342137 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.342110    5033 generic.go:356] "Generic (PLEG): container finished" podID="7827f4c0-6c4d-48e4-959a-47188b2693ab" containerID="6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9" exitCode=0
Oct 30 19:33:50.342241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.342158    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78455b774c-skgjw"
Oct 30 19:33:50.342241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.342169    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78455b774c-skgjw" event={"ID":"7827f4c0-6c4d-48e4-959a-47188b2693ab","Type":"ContainerDied","Data":"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"}
Oct 30 19:33:50.342241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.342216    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78455b774c-skgjw" event={"ID":"7827f4c0-6c4d-48e4-959a-47188b2693ab","Type":"ContainerDied","Data":"d18002924d393fb5ca2a082f5f42790f85b91cc00aa5c6ff16505c1a5168c9c5"}
Oct 30 19:33:50.342241 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.342239    5033 scope.go:122] "RemoveContainer" containerID="6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"
Oct 30 19:33:50.361578 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.361559    5033 scope.go:122] "RemoveContainer" containerID="6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"
Oct 30 19:33:50.361942 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:50.361920    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9\": container with ID starting with 6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9 not found: ID does not exist" containerID="6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"
Oct 30 19:33:50.362048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.361951    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9"} err="failed to get container status \"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9\": rpc error: code = NotFound desc = could not find container \"6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9\": container with ID starting with 6d2036685c19919e0764bf0a0b71c9453ac05fc4a1e8320d4e7114f9dc638cf9 not found: ID does not exist"
Oct 30 19:33:50.375169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.375143    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:33:50.380361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.380339    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78455b774c-skgjw"]
Oct 30 19:33:50.430743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.430720    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pws7h\" (UniqueName: \"kubernetes.io/projected/7827f4c0-6c4d-48e4-959a-47188b2693ab-kube-api-access-pws7h\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.430743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.430744    5033 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-webhook-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.430906 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:50.430753    5033 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7827f4c0-6c4d-48e4-959a-47188b2693ab-metrics-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:51.169047 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:51.169013    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7827f4c0-6c4d-48e4-959a-47188b2693ab" path="/var/lib/kubelet/pods/7827f4c0-6c4d-48e4-959a-47188b2693ab/volumes"
Oct 30 19:33:52.617401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:52.617044    5033 patch_prober.go:29] interesting pod/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:33:52.617401 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:52.617104    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 30 19:33:53.131735 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.131620    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201046    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201108    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201145    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201207    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201241    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201372    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201415    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201455    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.204701 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.201489    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\") pod \"79091d89-ccf8-489e-beaf-fcf36c91f684\" (UID: \"79091d89-ccf8-489e-beaf-fcf36c91f684\") "
Oct 30 19:33:53.246116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.245366    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.246116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.245585    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.246116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.245765    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.246116 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.245923    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.259898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.247843    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:53.259898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.249127    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:53.309696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.303821    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.335573 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.335525    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "kube-api-access-hxfwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339224    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy" pod "79091d89-ccf8-489e-beaf-fcf36c91f684" (UID: "79091d89-ccf8-489e-beaf-fcf36c91f684"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339280    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339309    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339325    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339338    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339352    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339367    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.339405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.339381    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79091d89-ccf8-489e-beaf-fcf36c91f684-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.440890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.440805    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxfwj\" (UniqueName: \"kubernetes.io/projected/79091d89-ccf8-489e-beaf-fcf36c91f684-kube-api-access-hxfwj\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.440890 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.440847    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79091d89-ccf8-489e-beaf-fcf36c91f684-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.445230 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.445201    5033 generic.go:356] "Generic (PLEG): container finished" podID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerID="f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af" exitCode=0
Oct 30 19:33:53.445347 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.445302    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"
Oct 30 19:33:53.445851 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.445822    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" event={"ID":"79091d89-ccf8-489e-beaf-fcf36c91f684","Type":"ContainerDied","Data":"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"}
Oct 30 19:33:53.446053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.446026    5033 scope.go:122] "RemoveContainer" containerID="f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"
Oct 30 19:33:53.446889 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.446859    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr" event={"ID":"79091d89-ccf8-489e-beaf-fcf36c91f684","Type":"ContainerDied","Data":"7a274d24582cce92a1f5eaff830f723e46b8177370aa9a839029aab5c76dd68d"}
Oct 30 19:33:53.449390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.449365    5033 generic.go:356] "Generic (PLEG): container finished" podID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerID="f6ca135896633d11d6fe96ad81313aa4eace07353e14ad42a60cffbf568672e5" exitCode=0
Oct 30 19:33:53.449513 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.449446    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" event={"ID":"cdde0ee1-e242-49f3-a81c-6241adfa007c","Type":"ContainerDied","Data":"f6ca135896633d11d6fe96ad81313aa4eace07353e14ad42a60cffbf568672e5"}
Oct 30 19:33:53.532696 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.532506    5033 generic.go:356] "Generic (PLEG): container finished" podID="834f5518-a544-487b-8c33-1148b603564c" containerID="5e6959dede6f7f644bc71661398d2243aa865a9e115866bb2eed1fc0e1f16338" exitCode=0
Oct 30 19:33:53.532855 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.532766    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" event={"ID":"834f5518-a544-487b-8c33-1148b603564c","Type":"ContainerDied","Data":"5e6959dede6f7f644bc71661398d2243aa865a9e115866bb2eed1fc0e1f16338"}
Oct 30 19:33:53.536308 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.536287    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:33:53.538756 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.538711    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:33:53.539460 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.539109    5033 scope.go:122] "RemoveContainer" containerID="f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"
Oct 30 19:33:53.539460 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:33:53.539385    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af\": container with ID starting with f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af not found: ID does not exist" containerID="f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"
Oct 30 19:33:53.539460 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.539421    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af"} err="failed to get container status \"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af\": rpc error: code = NotFound desc = could not find container \"f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af\": container with ID starting with f1d9d6c9b4ddeceba9e631e6104760842870bf54c193ed1992da8aa2fad8c3af not found: ID does not exist"
Oct 30 19:33:53.629548 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.629464    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:33:53.635638 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.635609    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-cbpvr"]
Oct 30 19:33:53.644559 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.644527    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.644816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.644794    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.645000 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.644982    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.645132 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645117    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.645265 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645250    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.645393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645378    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.645537 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645519    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645417    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645655    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645750    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645791    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645817    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645828    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645881    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645926    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645960    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.645986    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646033    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\") pod \"cdde0ee1-e242-49f3-a81c-6241adfa007c\" (UID: \"cdde0ee1-e242-49f3-a81c-6241adfa007c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646093    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646118    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\") pod \"834f5518-a544-487b-8c33-1148b603564c\" (UID: \"834f5518-a544-487b-8c33-1148b603564c\") "
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646490    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646507    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834f5518-a544-487b-8c33-1148b603564c-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646651    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.646978    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.647597    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.647826    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.647914    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.648332    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.648583    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:53.649458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.649422    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.675560 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.675520    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.682212 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.682166    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.682528 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.682492    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "kube-api-access-8hhxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.682932 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.682897    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.683118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.683099    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:53.683344 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.683326    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "kube-api-access-vbv8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:53.685305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.685272    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo" pod "cdde0ee1-e242-49f3-a81c-6241adfa007c" (UID: "cdde0ee1-e242-49f3-a81c-6241adfa007c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:53.687398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.687360    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy" pod "834f5518-a544-487b-8c33-1148b603564c" (UID: "834f5518-a544-487b-8c33-1148b603564c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748170    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748214    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748231    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748246    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834f5518-a544-487b-8c33-1148b603564c-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748260    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748274    5033 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748296    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748309    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834f5518-a544-487b-8c33-1148b603564c-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748325    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbv8w\" (UniqueName: \"kubernetes.io/projected/834f5518-a544-487b-8c33-1148b603564c-kube-api-access-vbv8w\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748341    5033 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-envoy\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748355    5033 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-workload-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748370    5033 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-credential-socket\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748384    5033 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-data\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748398    5033 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdde0ee1-e242-49f3-a81c-6241adfa007c-istio-podinfo\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748412    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hhxn\" (UniqueName: \"kubernetes.io/projected/cdde0ee1-e242-49f3-a81c-6241adfa007c-kube-api-access-8hhxn\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:53.748454 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:53.748428    5033 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdde0ee1-e242-49f3-a81c-6241adfa007c-istiod-ca-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:54.542294 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.542265    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"
Oct 30 19:33:54.542451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.542262    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj" event={"ID":"cdde0ee1-e242-49f3-a81c-6241adfa007c","Type":"ContainerDied","Data":"81052f046e5f189fff25342a080d412ec06e5fb896d4bd25cb0f445cd09278ba"}
Oct 30 19:33:54.542451 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.542390    5033 scope.go:122] "RemoveContainer" containerID="f6ca135896633d11d6fe96ad81313aa4eace07353e14ad42a60cffbf568672e5"
Oct 30 19:33:54.543800 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.543782    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"
Oct 30 19:33:54.543868 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.543822    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm" event={"ID":"834f5518-a544-487b-8c33-1148b603564c","Type":"ContainerDied","Data":"1a38bb007b50168a3e5c96adf4bedf78056f633f98d24fac9bc25d085d14db35"}
Oct 30 19:33:54.554069 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.554048    5033 scope.go:122] "RemoveContainer" containerID="5e6959dede6f7f644bc71661398d2243aa865a9e115866bb2eed1fc0e1f16338"
Oct 30 19:33:54.573496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.573448    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:33:54.581439 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.581414    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-kl4gj"]
Oct 30 19:33:54.595022 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.594992    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:33:54.600447 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:54.600424    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3bc085-openshift-default-77b977b9c5-ctzqm"]
Oct 30 19:33:55.168625 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:55.168586    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" path="/var/lib/kubelet/pods/79091d89-ccf8-489e-beaf-fcf36c91f684/volumes"
Oct 30 19:33:55.169232 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:55.169210    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="834f5518-a544-487b-8c33-1148b603564c" path="/var/lib/kubelet/pods/834f5518-a544-487b-8c33-1148b603564c/volumes"
Oct 30 19:33:55.169670 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:55.169652    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" path="/var/lib/kubelet/pods/cdde0ee1-e242-49f3-a81c-6241adfa007c/volumes"
Oct 30 19:33:55.293832 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:55.293800    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:33:55.295051 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:33:55.295027    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:34:04.986289 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:34:04.986253    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:34:31.032766 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:31.032733    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"]
Oct 30 19:34:31.040045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:31.040015    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a7202ab-l99pw"]
Oct 30 19:34:31.168053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:31.168024    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3f6a870-cce1-4ad3-9212-ae1cbf722794" path="/var/lib/kubelet/pods/d3f6a870-cce1-4ad3-9212-ae1cbf722794/volumes"
Oct 30 19:34:32.036029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:32.035997    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"]
Oct 30 19:34:32.039135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:32.039112    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e313cb88-dnmf7"]
Oct 30 19:34:33.167536 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:33.167504    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d" path="/var/lib/kubelet/pods/bd8edcdf-ddb6-4cb9-81e3-2719d0dd765d/volumes"
Oct 30 19:34:34.986076 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:34:34.986040    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:34:40.037255 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:40.037170    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"]
Oct 30 19:34:40.040148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:40.040122    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f57864d9-d8rql"]
Oct 30 19:34:41.172781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:41.172748    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fc201a1-c12d-4c5a-a738-a86972bb252b" path="/var/lib/kubelet/pods/1fc201a1-c12d-4c5a-a738-a86972bb252b/volumes"
Oct 30 19:34:44.028086 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:44.028052    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"]
Oct 30 19:34:44.032735 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:44.032705    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"]
Oct 30 19:34:44.035562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:44.035537    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a085c67d-h2lcb"]
Oct 30 19:34:44.038206 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:44.038183    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cf7b219e-lqrmn"]
Oct 30 19:34:45.170012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:45.169982    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77d7f8dc-2240-416a-ad87-59536dd9bf70" path="/var/lib/kubelet/pods/77d7f8dc-2240-416a-ad87-59536dd9bf70/volumes"
Oct 30 19:34:45.170400 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:45.170371    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5d93145-b8ef-4765-9bbf-887180ac277b" path="/var/lib/kubelet/pods/e5d93145-b8ef-4765-9bbf-887180ac277b/volumes"
Oct 30 19:34:53.028016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:53.027961    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"]
Oct 30 19:34:53.031421 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:53.031401    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acf6e151-2hxw9"]
Oct 30 19:34:53.168150 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:53.168121    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c4512b7-3617-4636-b82d-1fe489c9df4f" path="/var/lib/kubelet/pods/8c4512b7-3617-4636-b82d-1fe489c9df4f/volumes"
Oct 30 19:34:56.029786 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:56.029743    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"]
Oct 30 19:34:56.032498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:56.032476    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c6170d1-6hhmf"]
Oct 30 19:34:57.168052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:34:57.168021    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b479af8d-7727-4a7d-a1e7-2fa92f405a44" path="/var/lib/kubelet/pods/b479af8d-7727-4a7d-a1e7-2fa92f405a44/volumes"
Oct 30 19:35:05.046360 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:35:05.046329    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:35:09.046971 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:09.046935    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"]
Oct 30 19:35:09.055161 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:09.055135    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8cb6322e-hgvt6"]
Oct 30 19:35:09.167033 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:09.167005    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32a6c99b-ab1a-483c-861a-7049a68b0a07" path="/var/lib/kubelet/pods/32a6c99b-ab1a-483c-861a-7049a68b0a07/volumes"
Oct 30 19:35:23.043870 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:23.043833    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"]
Oct 30 19:35:23.048066 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:23.048045    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a3d5b3a9-m5bzw"]
Oct 30 19:35:23.168020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:23.167989    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b66f84b-3a5e-450c-9e9b-ee2e3e290628" path="/var/lib/kubelet/pods/5b66f84b-3a5e-450c-9e9b-ee2e3e290628/volumes"
Oct 30 19:35:34.986184 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:35:34.986148    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:35:55.909987 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.909959    5033 scope.go:122] "RemoveContainer" containerID="d5f96c38ea7240a6cb9cfdfebb3060681f2fdd39ad48706503f5ca8eb3f96248"
Oct 30 19:35:55.919712 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.919672    5033 scope.go:122] "RemoveContainer" containerID="fc574fa73a96db91b5fdaf3b1bb99b5a5296f5e0b5957f96bcd52f4024d6d3f5"
Oct 30 19:35:55.928050 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.928029    5033 scope.go:122] "RemoveContainer" containerID="a00172ffb2b0f6fe89940c6e242d128d7f6f3ed60dc7e5e4c7d331ae14b3b44a"
Oct 30 19:35:55.936092 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.936062    5033 scope.go:122] "RemoveContainer" containerID="83420c8a9c451db7e94dbce695399e9d5f64b28cededc0abfd099eaea23d1af8"
Oct 30 19:35:55.944147 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.944128    5033 scope.go:122] "RemoveContainer" containerID="3a6d5bf33e0c9ab4e16dedfd2738348684476a92847ba6de0f4de1d2605a29da"
Oct 30 19:35:55.953743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.953722    5033 scope.go:122] "RemoveContainer" containerID="230ba7f0cab01b2d129c90134ce03f8b8edc5dd4ec0c28c57aeba18495a10e18"
Oct 30 19:35:55.963141 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:35:55.963124    5033 scope.go:122] "RemoveContainer" containerID="5ceaea127ab017d5d67b3157417045a5c0f921d64d301df746f820f22a9ff5ec"
Oct 30 19:36:04.986148 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:36:04.986113    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:36:12.024562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:12.024474    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"]
Oct 30 19:36:12.026973 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:12.026948    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4a5a6fd7-6hx7n"]
Oct 30 19:36:13.173658 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:13.173223    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="549c5159-aa5b-42db-b11a-d725abeff58b" path="/var/lib/kubelet/pods/549c5159-aa5b-42db-b11a-d725abeff58b/volumes"
Oct 30 19:36:34.986051 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:36:34.986012    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:36:36.026289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:36.026256    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"]
Oct 30 19:36:36.029508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:36.029480    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12c19f23-tbv7m"]
Oct 30 19:36:37.167569 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:37.167535    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd" path="/var/lib/kubelet/pods/59dc7eeb-2934-45f2-bd1b-9c501ad3a2dd/volumes"
Oct 30 19:36:40.038738 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:40.038705    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"]
Oct 30 19:36:40.045173 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:40.045136    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b1467a7-2drhl"]
Oct 30 19:36:41.167798 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:41.167763    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c38fcf53-0871-4344-87f9-81bb0bdb3183" path="/var/lib/kubelet/pods/c38fcf53-0871-4344-87f9-81bb0bdb3183/volumes"
Oct 30 19:36:51.025042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:51.025006    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"]
Oct 30 19:36:51.029057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:51.029028    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec0d47cf-mxgb2"]
Oct 30 19:36:51.168346 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:51.168311    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4194440b-9220-4f02-b801-caabfe3b3034" path="/var/lib/kubelet/pods/4194440b-9220-4f02-b801-caabfe3b3034/volumes"
Oct 30 19:36:56.012704 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:56.012650    5033 scope.go:122] "RemoveContainer" containerID="213a4b803ffdd54f39dfdff2b0a1b872bfc6646263bdf0cf5470355477fa0506"
Oct 30 19:36:56.021687 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:56.021658    5033 scope.go:122] "RemoveContainer" containerID="0cc83cf141f43fce39fdc271cff310e3586188a0866600894e7886450011876c"
Oct 30 19:36:56.030243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:56.030223    5033 scope.go:122] "RemoveContainer" containerID="8e571525f9002409f6b3780758b3d7c92d053b0aebe49c5d42dffaa86ea7420d"
Oct 30 19:36:57.025612 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:57.025572    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"]
Oct 30 19:36:57.028911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:57.028885    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6ec888fd-4n54m"]
Oct 30 19:36:57.168060 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:36:57.168027    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e065156e-b44c-4892-a8c9-42e9a0a0efd7" path="/var/lib/kubelet/pods/e065156e-b44c-4892-a8c9-42e9a0a0efd7/volumes"
Oct 30 19:37:02.024350 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:02.024310    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"]
Oct 30 19:37:02.027848 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:02.027823    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b314d387-jsptn"]
Oct 30 19:37:03.167824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:03.167794    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91cc2627-ed53-406f-aafd-cf5fb847ee2b" path="/var/lib/kubelet/pods/91cc2627-ed53-406f-aafd-cf5fb847ee2b/volumes"
Oct 30 19:37:05.041701 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:37:05.041654    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:37:20.027317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:20.027280    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"]
Oct 30 19:37:20.032058 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:20.032028    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"]
Oct 30 19:37:20.034274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:20.034251    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38a69b09-n2vvc"]
Oct 30 19:37:20.037946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:20.037926    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d43671e5-z9h56"]
Oct 30 19:37:21.168057 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:21.168023    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52be9153-2a15-456c-a2ed-ae191b30ca25" path="/var/lib/kubelet/pods/52be9153-2a15-456c-a2ed-ae191b30ca25/volumes"
Oct 30 19:37:21.168532 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:21.168423    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7c84868-8e99-419e-96eb-31980f08f5ac" path="/var/lib/kubelet/pods/d7c84868-8e99-419e-96eb-31980f08f5ac/volumes"
Oct 30 19:37:24.032404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:24.032371    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"]
Oct 30 19:37:24.037746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:24.037725    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"]
Oct 30 19:37:24.040606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:24.040587    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75e08bdd-kqfxw"]
Oct 30 19:37:24.042440 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:24.042419    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2522e5ee-zzf7q"]
Oct 30 19:37:25.166638 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:25.166601    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06cc768f-89b3-4420-8664-e3f65b6ef227" path="/var/lib/kubelet/pods/06cc768f-89b3-4420-8664-e3f65b6ef227/volumes"
Oct 30 19:37:25.167103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:25.167087    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5afd0952-f8e4-4de3-975d-225a74f220f1" path="/var/lib/kubelet/pods/5afd0952-f8e4-4de3-975d-225a74f220f1/volumes"
Oct 30 19:37:27.025148 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:27.025086    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"]
Oct 30 19:37:27.030419 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:27.030392    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3ff1d931-znzh4"]
Oct 30 19:37:27.172515 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:27.172485    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0273b89-0592-4099-ad0a-93e4c5e4b0ad" path="/var/lib/kubelet/pods/f0273b89-0592-4099-ad0a-93e4c5e4b0ad/volumes"
Oct 30 19:37:34.028727 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:34.028664    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"]
Oct 30 19:37:34.031010 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:34.030982    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-09ef2027-sfmpb"]
Oct 30 19:37:34.986829 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:37:34.986794    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:37:35.174369 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:35.174328    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83d53f99-f75b-47fa-9870-8f28595e0344" path="/var/lib/kubelet/pods/83d53f99-f75b-47fa-9870-8f28595e0344/volumes"
Oct 30 19:37:37.025412 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:37.025326    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"]
Oct 30 19:37:37.027621 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:37.027591    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0851b971-vsmmg"]
Oct 30 19:37:37.169042 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:37.169005    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8533de32-5e10-4779-a285-89dd45de080e" path="/var/lib/kubelet/pods/8533de32-5e10-4779-a285-89dd45de080e/volumes"
Oct 30 19:37:43.040010 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.039974    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"]
Oct 30 19:37:43.042112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.042090    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"]
Oct 30 19:37:43.043736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.043725    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46a76b65-kw55x"]
Oct 30 19:37:43.045601 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.045570    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a1c28c5-nx5dl"]
Oct 30 19:37:43.167989 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.167962    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="709ff886-5f5a-48e9-8940-4dfd197c9a6f" path="/var/lib/kubelet/pods/709ff886-5f5a-48e9-8940-4dfd197c9a6f/volumes"
Oct 30 19:37:43.168377 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:43.168363    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78ce3811-25c2-4220-83a3-374f1c84248b" path="/var/lib/kubelet/pods/78ce3811-25c2-4220-83a3-374f1c84248b/volumes"
Oct 30 19:37:55.035496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:55.035459    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"]
Oct 30 19:37:55.038169 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:55.037621    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d9eef59-gdzb7"]
Oct 30 19:37:55.168215 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:55.168184    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a607509-8147-453f-a594-09a247f6ced5" path="/var/lib/kubelet/pods/0a607509-8147-453f-a594-09a247f6ced5/volumes"
Oct 30 19:37:56.063474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.063443    5033 scope.go:122] "RemoveContainer" containerID="ed5d92682f5744f5de313c1d09a005640d78a5a051e91355b79ec7570691e309"
Oct 30 19:37:56.072806 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.072757    5033 scope.go:122] "RemoveContainer" containerID="c84706125dc67da0043b8483dd85b82a8d61bcd1bb9fd636d1f6b10f2f865514"
Oct 30 19:37:56.080962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.080943    5033 scope.go:122] "RemoveContainer" containerID="5674d33a33ef96e993ba383e0092f25e981f2aaf43244fd39f69619d6014d5e4"
Oct 30 19:37:56.088960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.088940    5033 scope.go:122] "RemoveContainer" containerID="2f7b8e79a7de321e715c6706cb37f0a25cbf6630ed7cf91a613d795b0e6ca0e6"
Oct 30 19:37:56.096716 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.096696    5033 scope.go:122] "RemoveContainer" containerID="bb31177bd836de6e8feca5872ef778cd15ce31bf3d87af1b286c21c0ab7dc24e"
Oct 30 19:37:56.104281 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:37:56.104258    5033 scope.go:122] "RemoveContainer" containerID="034bab3d26dc39f031e1391260eee6a407119bac1593d7f3bb05b6171331c60c"
Oct 30 19:38:03.029825 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:03.029793    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"]
Oct 30 19:38:03.032941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:03.032913    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1395101c-hr6ft"]
Oct 30 19:38:03.166941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:03.166910    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4aea349b-f447-472d-8e0a-a6e9ca9e96f6" path="/var/lib/kubelet/pods/4aea349b-f447-472d-8e0a-a6e9ca9e96f6/volumes"
Oct 30 19:38:04.024248 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:04.024191    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"]
Oct 30 19:38:04.026418 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:04.026395    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d944f2b-hxhnf"]
Oct 30 19:38:04.985648 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:38:04.985615    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:38:05.167068 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:05.167041    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdb58caf-9f9d-43b1-ac07-0f402058348e" path="/var/lib/kubelet/pods/bdb58caf-9f9d-43b1-ac07-0f402058348e/volumes"
Oct 30 19:38:16.020655 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.013313    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:16.021317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.021282    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:16.225700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.224773    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:16.225700 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.224903    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:16.254797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.254759    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:16.255030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:16.254992    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:29.030693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:29.030642    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"]
Oct 30 19:38:29.041647 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:29.041609    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8151a70-gm9xz"]
Oct 30 19:38:29.171593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:29.171548    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd9e9034-bc0f-4b62-bec2-f02157fcbf78" path="/var/lib/kubelet/pods/bd9e9034-bc0f-4b62-bec2-f02157fcbf78/volumes"
Oct 30 19:38:31.018263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.017774    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-sx9vv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:31.018263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.017842    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-sx9vv" podUID="f0f928a1-6d04-4493-82ff-d83fda077a36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:31.219088 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.219041    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:31.219285 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.219102    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:31.244846 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.244808    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:31.245017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:31.244863    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:33.047397 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.047365    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:38:33.047824 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.047595    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" podUID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" containerName="discovery" containerID="cri-o://2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79" gracePeriod=30
Oct 30 19:38:33.305810 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.305758    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:38:33.419523 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419489    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.419733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419557    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.419733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419587    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.419733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419611    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.419733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419631    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.419733 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419685    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.420032 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.419736    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\") pod \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\" (UID: \"bc488364-b3a3-4802-adf0-f9211c1b3a6d\") "
Oct 30 19:38:33.420129 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.420090    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:33.421986 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.421955    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:33.422112 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.422013    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:33.422318 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.422291    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:33.422415 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.422312    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:33.422415 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.422346    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "kube-api-access-lcgkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:33.422415 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.422381    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert" pod "bc488364-b3a3-4802-adf0-f9211c1b3a6d" (UID: "bc488364-b3a3-4802-adf0-f9211c1b3a6d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:33.521151 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521112    5033 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-token\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521151 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521148    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcgkn\" (UniqueName: \"kubernetes.io/projected/bc488364-b3a3-4802-adf0-f9211c1b3a6d-kube-api-access-lcgkn\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521151 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521160    5033 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-cacerts\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521169    5033 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-ca-configmap\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521178    5033 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-kubeconfig\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521186    5033 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bc488364-b3a3-4802-adf0-f9211c1b3a6d-istio-csr-dns-cert\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.521387 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.521195    5033 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc488364-b3a3-4802-adf0-f9211c1b3a6d-local-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:33.627645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.627575    5033 generic.go:356] "Generic (PLEG): container finished" podID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" containerID="2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79" exitCode=0
Oct 30 19:38:33.627645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.627624    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"
Oct 30 19:38:33.627645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.627633    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" event={"ID":"bc488364-b3a3-4802-adf0-f9211c1b3a6d","Type":"ContainerDied","Data":"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"}
Oct 30 19:38:33.627895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.627672    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m" event={"ID":"bc488364-b3a3-4802-adf0-f9211c1b3a6d","Type":"ContainerDied","Data":"a6efce2af410904b24ebf79a1f73a9661ad4092ebb9d154f36779ec1f339aa3b"}
Oct 30 19:38:33.627895 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.627706    5033 scope.go:122] "RemoveContainer" containerID="2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"
Oct 30 19:38:33.637853 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.637838    5033 scope.go:122] "RemoveContainer" containerID="2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"
Oct 30 19:38:33.638128 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:38:33.638112    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79\": container with ID starting with 2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79 not found: ID does not exist" containerID="2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"
Oct 30 19:38:33.638172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.638135    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79"} err="failed to get container status \"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79\": rpc error: code = NotFound desc = could not find container \"2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79\": container with ID starting with 2b63e6c82dadd34d49e6ff2f689ebad97a9d6ff9951462bdf056dbb595e37e79 not found: ID does not exist"
Oct 30 19:38:33.654062 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.654041    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:38:33.655778 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:33.655760    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-h5d5m"]
Oct 30 19:38:34.986104 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:38:34.986070    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:38:35.171024 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:35.170993    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" path="/var/lib/kubelet/pods/bc488364-b3a3-4802-adf0-f9211c1b3a6d/volumes"
Oct 30 19:38:43.652335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.652284    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"]
Oct 30 19:38:43.653192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653163    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:38:43.653192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653190    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653215    5033 cpu_manager.go:414] "containerMap: removing container" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653223    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653264    5033 cpu_manager.go:414] "containerMap: removing container" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653271    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653288    5033 cpu_manager.go:414] "containerMap: removing container" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653296    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653317    5033 cpu_manager.go:414] "containerMap: removing container" podUID="7827f4c0-6c4d-48e4-959a-47188b2693ab" containerName="manager"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653324    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7827f4c0-6c4d-48e4-959a-47188b2693ab" containerName="manager"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653336    5033 cpu_manager.go:414] "containerMap: removing container" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653344    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653362    5033 cpu_manager.go:414] "containerMap: removing container" podUID="bd9e9034-bc0f-4b62-bec2-f02157fcbf78" containerName="revoke-keys"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653369    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd9e9034-bc0f-4b62-bec2-f02157fcbf78" containerName="revoke-keys"
Oct 30 19:38:43.653389 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653396    5033 cpu_manager.go:414] "containerMap: removing container" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653403    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653424    5033 cpu_manager.go:414] "containerMap: removing container" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653431    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653450    5033 cpu_manager.go:414] "containerMap: removing container" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653457    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653469    5033 cpu_manager.go:414] "containerMap: removing container" podUID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" containerName="discovery"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653475    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" containerName="discovery"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653492    5033 cpu_manager.go:414] "containerMap: removing container" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653499    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653508    5033 cpu_manager.go:414] "containerMap: removing container" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653515    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653717    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="9198743c-05d3-413a-a0b5-31ce2bf40c8a" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653732    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="db6c61fe-0de4-4c23-8753-d1a0b016a9a4" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653742    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="7827f4c0-6c4d-48e4-959a-47188b2693ab" containerName="manager"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653750    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdde0ee1-e242-49f3-a81c-6241adfa007c" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653768    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd9e9034-bc0f-4b62-bec2-f02157fcbf78" containerName="revoke-keys"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653784    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="79091d89-ccf8-489e-beaf-fcf36c91f684" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653793    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="54432de9-313b-4824-b1f3-be4ca88764ef" containerName="main"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653809    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="834f5518-a544-487b-8c33-1148b603564c" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653816    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653824    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc488364-b3a3-4802-adf0-f9211c1b3a6d" containerName="discovery"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653839    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="36c4f614-8b00-4946-8f5e-2df9d67b6d39" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653848    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="d34d5d61-7e02-4fcf-9fc9-178f8bc2f032" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653867    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:38:43.654118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.653878    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="359196ec-6ea5-4c10-ac93-2cb03f88a4ee" containerName="istio-proxy"
Oct 30 19:38:43.657924 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.657897    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.663770 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.663747    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"]
Oct 30 19:38:43.664402 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.664369    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:38:43.664582 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.664522    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:38:43.811777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.811744    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.811935 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.811801    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnvsb\" (UniqueName: \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.811994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.811936    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.912962 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.912897    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.913079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.912962    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.913079 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.912988    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnvsb\" (UniqueName: \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.913599 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.913571    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.915172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.915154    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.923228 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.923207    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnvsb\" (UniqueName: \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\") pod \"maas-api-revoke-sub-3d4156f1-8fsxc\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:43.981968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:43.981948    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:44.255563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:44.255536    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"]
Oct 30 19:38:44.256805 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:38:44.256774    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca30ff3_50f6_44ea_81e2_ada152de596a.slice/crio-8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc WatchSource:0}: Error finding container 8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc: Status 404 returned error can't find the container with id 8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc
Oct 30 19:38:44.703328 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:44.703294    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc" event={"ID":"3ca30ff3-50f6-44ea-81e2-ada152de596a","Type":"ContainerStarted","Data":"875abf5b21403adbe3e2ac8ff5bbb611b7a1e841a0c99aea6f294d85caf6c146"}
Oct 30 19:38:44.703781 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:44.703352    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc" event={"ID":"3ca30ff3-50f6-44ea-81e2-ada152de596a","Type":"ContainerStarted","Data":"8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc"}
Oct 30 19:38:44.724361 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:44.724315    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc" podStartSLOduration=1.7243028379999998 podStartE2EDuration="1.724302838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:44.723402182 +0000 UTC m=+1189.693043018" watchObservedRunningTime="2026-10-30 19:38:44.724302838 +0000 UTC m=+1189.693943673"
Oct 30 19:38:45.711950 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:45.711913    5033 generic.go:356] "Generic (PLEG): container finished" podID="3ca30ff3-50f6-44ea-81e2-ada152de596a" containerID="875abf5b21403adbe3e2ac8ff5bbb611b7a1e841a0c99aea6f294d85caf6c146" exitCode=0
Oct 30 19:38:45.712335 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:45.711994    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc" event={"ID":"3ca30ff3-50f6-44ea-81e2-ada152de596a","Type":"ContainerDied","Data":"875abf5b21403adbe3e2ac8ff5bbb611b7a1e841a0c99aea6f294d85caf6c146"}
Oct 30 19:38:46.853732 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:46.853707    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:47.038711 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.038607    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\") pod \"3ca30ff3-50f6-44ea-81e2-ada152de596a\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") "
Oct 30 19:38:47.038856 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.038747    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\") pod \"3ca30ff3-50f6-44ea-81e2-ada152de596a\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") "
Oct 30 19:38:47.038856 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.038767    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\" (UniqueName: \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\") pod \"3ca30ff3-50f6-44ea-81e2-ada152de596a\" (UID: \"3ca30ff3-50f6-44ea-81e2-ada152de596a\") "
Oct 30 19:38:47.039012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.038986    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca" pod "3ca30ff3-50f6-44ea-81e2-ada152de596a" (UID: "3ca30ff3-50f6-44ea-81e2-ada152de596a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:47.040671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.040650    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca" pod "3ca30ff3-50f6-44ea-81e2-ada152de596a" (UID: "3ca30ff3-50f6-44ea-81e2-ada152de596a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:47.040671 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.040657    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb" pod "3ca30ff3-50f6-44ea-81e2-ada152de596a" (UID: "3ca30ff3-50f6-44ea-81e2-ada152de596a"). InnerVolumeSpecName "kube-api-access-lnvsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:47.139433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.139402    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:47.139433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.139432    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnvsb\" (UniqueName: \"kubernetes.io/projected/3ca30ff3-50f6-44ea-81e2-ada152de596a-kube-api-access-lnvsb\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:47.139433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.139442    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ca30ff3-50f6-44ea-81e2-ada152de596a-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:47.723983 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.723958    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"
Oct 30 19:38:47.724135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.723995    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc" event={"ID":"3ca30ff3-50f6-44ea-81e2-ada152de596a","Type":"ContainerDied","Data":"8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc"}
Oct 30 19:38:47.724135 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:47.724040    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8308ab6c3039260b3a869770e460ecf261065ca15e95570472d66b89a43df4cc"
Oct 30 19:38:55.339815 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:55.339785    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:38:55.342044 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:55.342018    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:38:56.151524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.151494    5033 scope.go:122] "RemoveContainer" containerID="a1b9efbbc15392d0c3e5033bc556b4e9886aebfc6d37317104504bd80d468a0c"
Oct 30 19:38:56.160524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.160502    5033 scope.go:122] "RemoveContainer" containerID="adb53bedac2d07a40ddb53bde6d0f2b994c17a7f9f80a0631c4d7a85c4fd8f7d"
Oct 30 19:38:56.168493 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.168475    5033 scope.go:122] "RemoveContainer" containerID="cf3f76bca4a37c302ded3c2632514baf42a6a63b3349f7df1d9b51d75a3eeb01"
Oct 30 19:38:56.176077 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.176062    5033 scope.go:122] "RemoveContainer" containerID="aa5b5833887b5ba76103ad4ac054184bcf32b923ed38564aa9134331e47ba2f2"
Oct 30 19:38:56.183431 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.183415    5033 scope.go:122] "RemoveContainer" containerID="f1ac7797abb5fed9c64a5bb77a894ff03e2ddc19ef77adecb6c9cf22c51d1f81"
Oct 30 19:38:56.190947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.190929    5033 scope.go:122] "RemoveContainer" containerID="90bb24e3d861ceb6032c4fad546c74b796bbef34537191f96e5fa4acb8980db1"
Oct 30 19:38:56.198422 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.198407    5033 scope.go:122] "RemoveContainer" containerID="10b51867b409c47731ccebe67bcd03b4512cce20b01a1b1d6a6d610daa8ce6a2"
Oct 30 19:38:56.205652 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.205637    5033 scope.go:122] "RemoveContainer" containerID="556dbef4bd84efa89bd942005e08dcb2c92cb00911a86d8524e9faaacc856e38"
Oct 30 19:38:56.213243 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.213224    5033 scope.go:122] "RemoveContainer" containerID="aaaac34d5db2b6a58a289ed853eda1603afecde4d9afb6b8bebc87359fc24571"
Oct 30 19:38:56.220608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.220592    5033 scope.go:122] "RemoveContainer" containerID="f018b5f6f6ac052471c15adbe9f31fb96e60fc0d5cdeca387424261a02bc664c"
Oct 30 19:38:56.227917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.227899    5033 scope.go:122] "RemoveContainer" containerID="40008faa3158458be2e923e54c7b282cc0d01585fb3227c779ffed15cdf755c0"
Oct 30 19:38:56.235320 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.235303    5033 scope.go:122] "RemoveContainer" containerID="b1466db9a7d101a94aef43c3d5056083e631b3976f305f381bc47072c066565d"
Oct 30 19:38:56.242642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:38:56.242624    5033 scope.go:122] "RemoveContainer" containerID="85a0fa88e08f6d9a6f3f52382547bf4dccf13a44112fbbcd4cd78cb113b6cb98"
Oct 30 19:39:04.986421 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:39:04.986378    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:39:15.430504 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.430415    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:39:15.431127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.431067    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3ca30ff3-50f6-44ea-81e2-ada152de596a" containerName="revoke-keys"
Oct 30 19:39:15.431127 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.431089    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ca30ff3-50f6-44ea-81e2-ada152de596a" containerName="revoke-keys"
Oct 30 19:39:15.431303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.431171    5033 cpu_manager.go:414] "containerMap: removing container" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:39:15.431303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.431182    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e029a6d-6838-4509-a872-fe1dbb27b219" containerName="istio-proxy"
Oct 30 19:39:15.431303 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.431287    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ca30ff3-50f6-44ea-81e2-ada152de596a" containerName="revoke-keys"
Oct 30 19:39:15.434455 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.434424    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.450730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.450696    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:39:15.451814 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.451790    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-88ec2c\"/\"default-dockercfg-2xgbj\""
Oct 30 19:39:15.451946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.451832    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-88ec2c\"/\"kube-root-ca.crt\""
Oct 30 19:39:15.451946 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.451868    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-88ec2c\"/\"simulator-single-main-88ec2c-kserve-self-signed-certs\""
Oct 30 19:39:15.452039 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.451954    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-88ec2c\"/\"openshift-service-ca.crt\""
Oct 30 19:39:15.495247 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495219    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.495382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495263    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx94s\" (UniqueName: \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.495382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495337    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.495382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495361    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.495524 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495386    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.495563 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.495529    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596549 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596516    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596576    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx94s\" (UniqueName: \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596611    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596634    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596670    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.596753 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.596749    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.597230 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.597207    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.597472 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.597454    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.598286 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.598262    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.600406 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.600381    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.603498 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.603475    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.608864 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.608840    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx94s\" (UniqueName: \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\") pod \"simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") " pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:15.758452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:15.758421    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:16.046526 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:16.046499    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:39:16.048924 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:39:16.048894    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbba1fa32_6847_4808_91bb_21f46009e005.slice/crio-a5d5e71b5af34956095f7811087aeca5cdaf666bf58b354af2ba05ad58203b6c WatchSource:0}: Error finding container a5d5e71b5af34956095f7811087aeca5cdaf666bf58b354af2ba05ad58203b6c: Status 404 returned error can't find the container with id a5d5e71b5af34956095f7811087aeca5cdaf666bf58b354af2ba05ad58203b6c
Oct 30 19:39:16.883105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:16.883067    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" event={"ID":"bba1fa32-6847-4808-91bb-21f46009e005","Type":"ContainerStarted","Data":"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"}
Oct 30 19:39:16.883105 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:16.883111    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" event={"ID":"bba1fa32-6847-4808-91bb-21f46009e005","Type":"ContainerStarted","Data":"a5d5e71b5af34956095f7811087aeca5cdaf666bf58b354af2ba05ad58203b6c"}
Oct 30 19:39:16.904511 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:16.904464    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" podStartSLOduration=1.90445189 podStartE2EDuration="1.90445189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:16.902708041 +0000 UTC m=+1221.872348877" watchObservedRunningTime="2026-10-30 19:39:16.90445189 +0000 UTC m=+1221.874092726"
Oct 30 19:39:17.759352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:17.759321    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:25.759596 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:25.759559    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:25.771940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:25.771918    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:25.945504 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:25.945472    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:39:34.990393 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:39:34.990346    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:39:56.314445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:56.314410    5033 scope.go:122] "RemoveContainer" containerID="109167b65dd853abab096c51dadfcb49b758040764b662d65d5bcaffa95c871f"
Oct 30 19:39:56.323663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:56.323639    5033 scope.go:122] "RemoveContainer" containerID="aea92ca2c692c35255b5db24a16b239a446d55d858bec387efbfc8810479f614"
Oct 30 19:39:56.331349 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:56.331328    5033 scope.go:122] "RemoveContainer" containerID="c3deea7160bb623fc2606b8354bc54ca929414d4a46a52ca979710bcaca61cea"
Oct 30 19:39:56.338969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:39:56.338950    5033 scope.go:122] "RemoveContainer" containerID="c4a78ffcbe64f32d14807962e281ca3c5a03054d09fad44eb153a12abf4f2b77"
Oct 30 19:40:00.544379 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:00.544341    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:40:04.986850 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:40:04.986807    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:40:14.857772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:14.857737    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"]
Oct 30 19:40:14.862826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:14.862798    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-8fsxc"]
Oct 30 19:40:15.168831 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.168748    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ca30ff3-50f6-44ea-81e2-ada152de596a" path="/var/lib/kubelet/pods/3ca30ff3-50f6-44ea-81e2-ada152de596a/volumes"
Oct 30 19:40:15.564743 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.564685    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" podUID="bba1fa32-6847-4808-91bb-21f46009e005" containerName="main" containerID="cri-o://3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056" gracePeriod=45
Oct 30 19:40:15.812628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.812601    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:40:15.861496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861428    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.861496 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861478    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861502    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861543    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\" (UniqueName: \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861566    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861600    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\" (UniqueName: \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\") pod \"bba1fa32-6847-4808-91bb-21f46009e005\" (UID: \"bba1fa32-6847-4808-91bb-21f46009e005\") "
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861764    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861849    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:15.862001 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.861860    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:15.862388 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.862234    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:15.862388 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.862257    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:15.862388 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.862270    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:15.864005 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.863973    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:15.864124 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.864006    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "kube-api-access-cx94s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:15.864124 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.864030    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm" pod "bba1fa32-6847-4808-91bb-21f46009e005" (UID: "bba1fa32-6847-4808-91bb-21f46009e005"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:15.962941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.962909    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba1fa32-6847-4808-91bb-21f46009e005-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:15.962941 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.962936    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx94s\" (UniqueName: \"kubernetes.io/projected/bba1fa32-6847-4808-91bb-21f46009e005-kube-api-access-cx94s\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:15.963123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:15.962949    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba1fa32-6847-4808-91bb-21f46009e005-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:16.211628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.211594    5033 generic.go:356] "Generic (PLEG): container finished" podID="bba1fa32-6847-4808-91bb-21f46009e005" containerID="3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056" exitCode=0
Oct 30 19:40:16.211797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.211640    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"
Oct 30 19:40:16.211797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.211697    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" event={"ID":"bba1fa32-6847-4808-91bb-21f46009e005","Type":"ContainerDied","Data":"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"}
Oct 30 19:40:16.211797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.211739    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch" event={"ID":"bba1fa32-6847-4808-91bb-21f46009e005","Type":"ContainerDied","Data":"a5d5e71b5af34956095f7811087aeca5cdaf666bf58b354af2ba05ad58203b6c"}
Oct 30 19:40:16.211797 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.211757    5033 scope.go:122] "RemoveContainer" containerID="3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"
Oct 30 19:40:16.225323 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.225305    5033 scope.go:122] "RemoveContainer" containerID="3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"
Oct 30 19:40:16.225914 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:40:16.225886    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056\": container with ID starting with 3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056 not found: ID does not exist" containerID="3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"
Oct 30 19:40:16.226011 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.225923    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056"} err="failed to get container status \"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056\": rpc error: code = NotFound desc = could not find container \"3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056\": container with ID starting with 3ecdcb7b7b4d8b0f36e63959f75a32fa0d5e59089bfd7fc3324fb130c8ea2056 not found: ID does not exist"
Oct 30 19:40:16.237056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.237035    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:40:16.243299 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:16.243280    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-88ec2c/simulator-single-main-88ec2c-kserve-5fdc4c8dd5-l79ch"]
Oct 30 19:40:17.167754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:17.167726    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba1fa32-6847-4808-91bb-21f46009e005" path="/var/lib/kubelet/pods/bba1fa32-6847-4808-91bb-21f46009e005/volumes"
Oct 30 19:40:33.662969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:33.662927    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:40:33.663593 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:33.663182    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" podUID="b6a72a87-9a52-4842-8c20-257e72176f15" containerName="manager" containerID="cri-o://6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433" gracePeriod=10
Oct 30 19:40:33.920445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:33.920390    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:40:34.040840 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.040810    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\") pod \"b6a72a87-9a52-4842-8c20-257e72176f15\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") "
Oct 30 19:40:34.041007 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.040855    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\" (UniqueName: \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\") pod \"b6a72a87-9a52-4842-8c20-257e72176f15\" (UID: \"b6a72a87-9a52-4842-8c20-257e72176f15\") "
Oct 30 19:40:34.041268 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.041241    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume" pod "b6a72a87-9a52-4842-8c20-257e72176f15" (UID: "b6a72a87-9a52-4842-8c20-257e72176f15"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:34.042931 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.042903    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg" pod "b6a72a87-9a52-4842-8c20-257e72176f15" (UID: "b6a72a87-9a52-4842-8c20-257e72176f15"). InnerVolumeSpecName "kube-api-access-l74rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:34.141620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.141595    5033 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a72a87-9a52-4842-8c20-257e72176f15-extensions-socket-volume\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.141620 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.141619    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l74rg\" (UniqueName: \"kubernetes.io/projected/b6a72a87-9a52-4842-8c20-257e72176f15-kube-api-access-l74rg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:34.333297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.333271    5033 generic.go:356] "Generic (PLEG): container finished" podID="b6a72a87-9a52-4842-8c20-257e72176f15" containerID="6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433" exitCode=0
Oct 30 19:40:34.333449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.333326    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"
Oct 30 19:40:34.333449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.333338    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" event={"ID":"b6a72a87-9a52-4842-8c20-257e72176f15","Type":"ContainerDied","Data":"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"}
Oct 30 19:40:34.333449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.333389    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2" event={"ID":"b6a72a87-9a52-4842-8c20-257e72176f15","Type":"ContainerDied","Data":"9ca76f980e1d4c8e916bdba9028ee9208009159a34d02b9459829e744d9f2c3f"}
Oct 30 19:40:34.333449 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.333419    5033 scope.go:122] "RemoveContainer" containerID="6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"
Oct 30 19:40:34.344584 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.344564    5033 scope.go:122] "RemoveContainer" containerID="6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"
Oct 30 19:40:34.344867 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:40:34.344848    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433\": container with ID starting with 6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433 not found: ID does not exist" containerID="6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"
Oct 30 19:40:34.344961 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.344878    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433"} err="failed to get container status \"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433\": rpc error: code = NotFound desc = could not find container \"6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433\": container with ID starting with 6dd8f095840da89a8763446938eb3adc803e0b0ca8ccbfce76cd2cb55e737433 not found: ID does not exist"
Oct 30 19:40:34.357814 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.357793    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:40:34.362095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:34.362073    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pjpn2"]
Oct 30 19:40:34.985313 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:40:34.985279    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:40:35.167504 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:40:35.167471    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6a72a87-9a52-4842-8c20-257e72176f15" path="/var/lib/kubelet/pods/b6a72a87-9a52-4842-8c20-257e72176f15/volumes"
Oct 30 19:41:04.985588 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:41:04.985553    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:41:34.731060 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731027    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"]
Oct 30 19:41:34.731435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731403    5033 cpu_manager.go:414] "containerMap: removing container" podUID="bba1fa32-6847-4808-91bb-21f46009e005" containerName="main"
Oct 30 19:41:34.731435 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731415    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba1fa32-6847-4808-91bb-21f46009e005" containerName="main"
Oct 30 19:41:34.731514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731447    5033 cpu_manager.go:414] "containerMap: removing container" podUID="b6a72a87-9a52-4842-8c20-257e72176f15" containerName="manager"
Oct 30 19:41:34.731514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731453    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6a72a87-9a52-4842-8c20-257e72176f15" containerName="manager"
Oct 30 19:41:34.731514 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731513    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6a72a87-9a52-4842-8c20-257e72176f15" containerName="manager"
Oct 30 19:41:34.731608 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.731523    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba1fa32-6847-4808-91bb-21f46009e005" containerName="main"
Oct 30 19:41:34.734631 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.734615    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.741724 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.741704    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vlzxc\""
Oct 30 19:41:34.757703 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.757662    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"]
Oct 30 19:41:34.846183 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.846156    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b68ff9f8-167b-4de0-9122-7b30984755c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.846296 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.846187    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtjz\" (UniqueName: \"kubernetes.io/projected/b68ff9f8-167b-4de0-9122-7b30984755c8-kube-api-access-5qtjz\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.946726 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.946671    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b68ff9f8-167b-4de0-9122-7b30984755c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.946844 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.946737    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtjz\" (UniqueName: \"kubernetes.io/projected/b68ff9f8-167b-4de0-9122-7b30984755c8-kube-api-access-5qtjz\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.947056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.947033    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b68ff9f8-167b-4de0-9122-7b30984755c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.956898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:34.956875    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtjz\" (UniqueName: \"kubernetes.io/projected/b68ff9f8-167b-4de0-9122-7b30984755c8-kube-api-access-5qtjz\") pod \"kuadrant-operator-controller-manager-678d87766f-qwtdd\" (UID: \"b68ff9f8-167b-4de0-9122-7b30984755c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:34.985810 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:41:34.985751    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:41:35.047134 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:35.047114    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:35.331693 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:35.331644    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"]
Oct 30 19:41:35.332248 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:41:35.332223    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68ff9f8_167b_4de0_9122_7b30984755c8.slice/crio-304fa2c248eb8474fa58114fe2a60a208ca7f5111243081e07110150350479c2 WatchSource:0}: Error finding container 304fa2c248eb8474fa58114fe2a60a208ca7f5111243081e07110150350479c2: Status 404 returned error can't find the container with id 304fa2c248eb8474fa58114fe2a60a208ca7f5111243081e07110150350479c2
Oct 30 19:41:35.628172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:35.628081    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd" event={"ID":"b68ff9f8-167b-4de0-9122-7b30984755c8","Type":"ContainerStarted","Data":"78306eb1384fa0d76e0e049c0968c0488a686be7f9b67eb672f1fe5cd8f4bb0a"}
Oct 30 19:41:35.628172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:35.628142    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd" event={"ID":"b68ff9f8-167b-4de0-9122-7b30984755c8","Type":"ContainerStarted","Data":"304fa2c248eb8474fa58114fe2a60a208ca7f5111243081e07110150350479c2"}
Oct 30 19:41:35.628172 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:35.628172    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:46.635470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:46.635441    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd"
Oct 30 19:41:46.663759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:46.663711    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-qwtdd" podStartSLOduration=12.663699607 podStartE2EDuration="12.663699607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:35.668306104 +0000 UTC m=+1360.637946936" watchObservedRunningTime="2026-10-30 19:41:46.663699607 +0000 UTC m=+1371.633340441"
Oct 30 19:41:56.175910 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.175873    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"]
Oct 30 19:41:56.179959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.179939    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.185968 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.185941    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:41:56.186091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.186072    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:41:56.192459 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.192436    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"]
Oct 30 19:41:56.249917 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.249889    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.250030 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.249929    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsnbg\" (UniqueName: \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.250192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.250177    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.351048 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.351017    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.351179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.351077    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.351179 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.351102    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsnbg\" (UniqueName: \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.351759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.351738    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.353404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.353385    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.361828 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.361803    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsnbg\" (UniqueName: \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\") pod \"maas-api-revoke-sub-d576e68b-spwxg\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.494685 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.494655    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:56.766495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:56.766473    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"]
Oct 30 19:41:56.768024 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:41:56.767998    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d20ee7_0778_4cca_93ed_95e7c598f581.slice/crio-60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded WatchSource:0}: Error finding container 60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded: Status 404 returned error can't find the container with id 60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded
Oct 30 19:41:57.759926 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:57.759888    5033 generic.go:356] "Generic (PLEG): container finished" podID="99d20ee7-0778-4cca-93ed-95e7c598f581" containerID="d4674428148609449a5d2cd1de0791df5dcbbef850fc94bd348b1359c45ea19e" exitCode=0
Oct 30 19:41:57.760337 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:57.759973    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg" event={"ID":"99d20ee7-0778-4cca-93ed-95e7c598f581","Type":"ContainerDied","Data":"d4674428148609449a5d2cd1de0791df5dcbbef850fc94bd348b1359c45ea19e"}
Oct 30 19:41:57.760337 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:57.760026    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg" event={"ID":"99d20ee7-0778-4cca-93ed-95e7c598f581","Type":"ContainerStarted","Data":"60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded"}
Oct 30 19:41:58.900858 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.900829    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:58.976103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.976080    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\") pod \"99d20ee7-0778-4cca-93ed-95e7c598f581\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") "
Oct 30 19:41:58.976242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.976114    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\" (UniqueName: \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\") pod \"99d20ee7-0778-4cca-93ed-95e7c598f581\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") "
Oct 30 19:41:58.976242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.976180    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\") pod \"99d20ee7-0778-4cca-93ed-95e7c598f581\" (UID: \"99d20ee7-0778-4cca-93ed-95e7c598f581\") "
Oct 30 19:41:58.976423 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.976400    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca" pod "99d20ee7-0778-4cca-93ed-95e7c598f581" (UID: "99d20ee7-0778-4cca-93ed-95e7c598f581"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:58.978261 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.978239    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg" pod "99d20ee7-0778-4cca-93ed-95e7c598f581" (UID: "99d20ee7-0778-4cca-93ed-95e7c598f581"). InnerVolumeSpecName "kube-api-access-qsnbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:58.978348 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:58.978261    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca" pod "99d20ee7-0778-4cca-93ed-95e7c598f581" (UID: "99d20ee7-0778-4cca-93ed-95e7c598f581"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:59.077070 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.077016    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:59.077070 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.077039    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsnbg\" (UniqueName: \"kubernetes.io/projected/99d20ee7-0778-4cca-93ed-95e7c598f581-kube-api-access-qsnbg\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:59.077070 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.077050    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99d20ee7-0778-4cca-93ed-95e7c598f581-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:59.771782 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.771752    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"
Oct 30 19:41:59.771928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.771780    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg" event={"ID":"99d20ee7-0778-4cca-93ed-95e7c598f581","Type":"ContainerDied","Data":"60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded"}
Oct 30 19:41:59.771928 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:41:59.771828    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60ad7fd3fb15aed85a2d5dcbcb2a0e80f2dcaac538ebd8517c2e39e274d81ded"
Oct 30 19:42:04.986120 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:42:04.986083    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:42:15.172633 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.172493    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"]
Oct 30 19:42:15.173084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.173005    5033 cpu_manager.go:414] "containerMap: removing container" podUID="99d20ee7-0778-4cca-93ed-95e7c598f581" containerName="revoke-keys"
Oct 30 19:42:15.173084 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.173024    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99d20ee7-0778-4cca-93ed-95e7c598f581" containerName="revoke-keys"
Oct 30 19:42:15.173180 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.173164    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="99d20ee7-0778-4cca-93ed-95e7c598f581" containerName="revoke-keys"
Oct 30 19:42:15.176209 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.176191    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.185013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.184997    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:42:15.185566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.185548    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:42:15.189288 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.189264    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"]
Oct 30 19:42:15.224316 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.224277    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrzl\" (UniqueName: \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.224398 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.224381    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.224495 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.224478    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.325272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.325247    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.325365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.325331    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrzl\" (UniqueName: \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.325405 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.325387    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.325891 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.325872    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.327647 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.327626    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.336053 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.336034    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrzl\" (UniqueName: \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\") pod \"maas-api-revoke-sub-d2bc93bc-mcnkq\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.489777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.489757    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:15.766718 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.766692    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"]
Oct 30 19:42:15.769382 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:42:15.769353    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0f9d20_4476_4051_b9a2_3e3a945b9337.slice/crio-1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370 WatchSource:0}: Error finding container 1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370: Status 404 returned error can't find the container with id 1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370
Oct 30 19:42:15.891274 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:15.891243    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq" event={"ID":"4c0f9d20-4476-4051-b9a2-3e3a945b9337","Type":"ContainerStarted","Data":"1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370"}
Oct 30 19:42:16.897651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:16.897589    5033 generic.go:356] "Generic (PLEG): container finished" podID="4c0f9d20-4476-4051-b9a2-3e3a945b9337" containerID="b4b4dc603ea5a6a42b4714edaf29337a9b92ff14c94b01b250433cff4df70e91" exitCode=0
Oct 30 19:42:16.897651 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:16.897640    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq" event={"ID":"4c0f9d20-4476-4051-b9a2-3e3a945b9337","Type":"ContainerDied","Data":"b4b4dc603ea5a6a42b4714edaf29337a9b92ff14c94b01b250433cff4df70e91"}
Oct 30 19:42:18.046463 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.046440    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:18.149474 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.149450    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\" (UniqueName: \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\") pod \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") "
Oct 30 19:42:18.149604 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.149573    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\") pod \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") "
Oct 30 19:42:18.149645 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.149604    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\") pod \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\" (UID: \"4c0f9d20-4476-4051-b9a2-3e3a945b9337\") "
Oct 30 19:42:18.149969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.149946    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca" pod "4c0f9d20-4476-4051-b9a2-3e3a945b9337" (UID: "4c0f9d20-4476-4051-b9a2-3e3a945b9337"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:18.151622 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.151594    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca" pod "4c0f9d20-4476-4051-b9a2-3e3a945b9337" (UID: "4c0f9d20-4476-4051-b9a2-3e3a945b9337"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:18.151755 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.151595    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl" pod "4c0f9d20-4476-4051-b9a2-3e3a945b9337" (UID: "4c0f9d20-4476-4051-b9a2-3e3a945b9337"). InnerVolumeSpecName "kube-api-access-9lrzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:18.250399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.250377    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-openshift-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:18.250399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.250399    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0f9d20-4476-4051-b9a2-3e3a945b9337-maas-api-xks-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:18.250529 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.250410    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lrzl\" (UniqueName: \"kubernetes.io/projected/4c0f9d20-4476-4051-b9a2-3e3a945b9337-kube-api-access-9lrzl\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:18.917012 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.916984    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"
Oct 30 19:42:18.917206 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.917015    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq" event={"ID":"4c0f9d20-4476-4051-b9a2-3e3a945b9337","Type":"ContainerDied","Data":"1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370"}
Oct 30 19:42:18.917206 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:18.917060    5033 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e99099ec212e2b4aca9d018df0a3f1ffacc406ba7bdd0705cc443bcceec0370"
Oct 30 19:42:34.986202 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:42:34.986155    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:42:48.837192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.837159    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:42:48.837721 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.837702    5033 cpu_manager.go:414] "containerMap: removing container" podUID="4c0f9d20-4476-4051-b9a2-3e3a945b9337" containerName="revoke-keys"
Oct 30 19:42:48.837780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.837722    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c0f9d20-4476-4051-b9a2-3e3a945b9337" containerName="revoke-keys"
Oct 30 19:42:48.837816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.837806    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c0f9d20-4476-4051-b9a2-3e3a945b9337" containerName="revoke-keys"
Oct 30 19:42:48.840897 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.840873    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.855365 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.855342    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-ad6b27\"/\"default-dockercfg-4jmr6\""
Oct 30 19:42:48.855503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.855445    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-ad6b27\"/\"simulator-single-main-ad6b27-kserve-self-signed-certs\""
Oct 30 19:42:48.855503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.855455    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-ad6b27\"/\"openshift-service-ca.crt\""
Oct 30 19:42:48.855882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.855860    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-ad6b27\"/\"kube-root-ca.crt\""
Oct 30 19:42:48.882471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.882443    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:42:48.923754 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.923711    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.923906 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.923838    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gl5m\" (UniqueName: \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.923959 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.923921    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.924015 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.923996    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.924100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.924085    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:48.924153 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:48.924135    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025233 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025202    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025256    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025288    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025338    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025542    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gl5m\" (UniqueName: \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025595    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025664    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025854 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025729    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.025966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.025947    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.028218 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.028179    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.028562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.028542    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.036289 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.036257    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gl5m\" (UniqueName: \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\") pod \"simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") " pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.161191 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.161096    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:49.444849 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:49.444823    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:42:49.446449 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:42:49.446422    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c4a7dfb_7460_4f12_af2f_3ac8eda41498.slice/crio-b8abc2b6eaad29246559a1c73cef01b5d6a931a1d41f274f4a67ac59112771ab WatchSource:0}: Error finding container b8abc2b6eaad29246559a1c73cef01b5d6a931a1d41f274f4a67ac59112771ab: Status 404 returned error can't find the container with id b8abc2b6eaad29246559a1c73cef01b5d6a931a1d41f274f4a67ac59112771ab
Oct 30 19:42:50.085396 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:50.085365    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" event={"ID":"5c4a7dfb-7460-4f12-af2f-3ac8eda41498","Type":"ContainerStarted","Data":"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"}
Oct 30 19:42:50.085882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:50.085409    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" event={"ID":"5c4a7dfb-7460-4f12-af2f-3ac8eda41498","Type":"ContainerStarted","Data":"b8abc2b6eaad29246559a1c73cef01b5d6a931a1d41f274f4a67ac59112771ab"}
Oct 30 19:42:50.130408 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:50.130361    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" podStartSLOduration=2.130349219 podStartE2EDuration="2.130349219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:50.127857941 +0000 UTC m=+1435.097498776" watchObservedRunningTime="2026-10-30 19:42:50.130349219 +0000 UTC m=+1435.099990053"
Oct 30 19:42:50.161319 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:50.161294    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:59.161252 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:59.161216    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:42:59.174167 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:42:59.174143    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:43:00.147913 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:00.147884    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:43:04.987069 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:43:04.987037    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:43:33.503960 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:33.503921    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:43:34.986778 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:43:34.986740    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:43:48.523911 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.523862    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" podUID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" containerName="main" containerID="cri-o://4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae" gracePeriod=45
Oct 30 19:43:48.799234 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.799207    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:43:48.896412 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896380    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896423    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896440    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896495    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\" (UniqueName: \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896531    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896606 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896603    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\") pod \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\" (UID: \"5c4a7dfb-7460-4f12-af2f-3ac8eda41498\") "
Oct 30 19:43:48.896915 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896715    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:48.896976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896911    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:48.896976 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.896872    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:48.897192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.897168    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:48.897192 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.897195    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:48.897399 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.897207    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:48.898845 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.898817    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:48.899095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.899067    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:48.899095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.899087    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m" pod "5c4a7dfb-7460-4f12-af2f-3ac8eda41498" (UID: "5c4a7dfb-7460-4f12-af2f-3ac8eda41498"). InnerVolumeSpecName "kube-api-access-8gl5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:48.998357 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.998326    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gl5m\" (UniqueName: \"kubernetes.io/projected/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-kube-api-access-8gl5m\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:48.998357 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.998351    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:48.998357 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:48.998360    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4a7dfb-7460-4f12-af2f-3ac8eda41498-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:49.402505 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.402460    5033 generic.go:356] "Generic (PLEG): container finished" podID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" containerID="4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae" exitCode=0
Oct 30 19:43:49.402669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.402524    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"
Oct 30 19:43:49.402669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.402546    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" event={"ID":"5c4a7dfb-7460-4f12-af2f-3ac8eda41498","Type":"ContainerDied","Data":"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"}
Oct 30 19:43:49.402669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.402604    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg" event={"ID":"5c4a7dfb-7460-4f12-af2f-3ac8eda41498","Type":"ContainerDied","Data":"b8abc2b6eaad29246559a1c73cef01b5d6a931a1d41f274f4a67ac59112771ab"}
Oct 30 19:43:49.402669 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.402631    5033 scope.go:122] "RemoveContainer" containerID="4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"
Oct 30 19:43:49.412045 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.412023    5033 scope.go:122] "RemoveContainer" containerID="4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"
Oct 30 19:43:49.412322 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:43:49.412302    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae\": container with ID starting with 4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae not found: ID does not exist" containerID="4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"
Oct 30 19:43:49.412370 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.412331    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae"} err="failed to get container status \"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae\": rpc error: code = NotFound desc = could not find container \"4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae\": container with ID starting with 4e6fac63dc61b9e213c00b56d1381fe0ba1c03edb737b2e2d863ec15f8e654ae not found: ID does not exist"
Oct 30 19:43:49.433562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.433512    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:43:49.439404 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:49.439384    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-ad6b27/simulator-single-main-ad6b27-kserve-846d99d48d-5pzbg"]
Oct 30 19:43:51.167642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:51.167611    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" path="/var/lib/kubelet/pods/5c4a7dfb-7460-4f12-af2f-3ac8eda41498/volumes"
Oct 30 19:43:55.389114 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:55.389079    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:43:55.410095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:43:55.410064    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:44:04.985947 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:44:04.985909    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:44:12.196363 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.196329    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:12.196791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.196752    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" containerName="main"
Oct 30 19:44:12.196791 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.196766    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" containerName="main"
Oct 30 19:44:12.196878 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.196861    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c4a7dfb-7460-4f12-af2f-3ac8eda41498" containerName="main"
Oct 30 19:44:12.201562 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.201541    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.221032 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.221006    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 19:44:12.222415 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.222390    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-56e018f0-kserve-self-signed-certs\""
Oct 30 19:44:12.222590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.222477    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 19:44:12.222590 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.222520    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ctlhc\""
Oct 30 19:44:12.223729 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.223705    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:12.331834 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.331802    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.332029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.331879    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.332029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.331908    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.332159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.332039    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.332159 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.332083    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.332263 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.332196    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vvk5\" (UniqueName: \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433272 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433239    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433281    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433317    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vvk5\" (UniqueName: \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433357    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433445 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433407    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433449    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.433829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433807    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.434017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433954    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.434017 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.433980    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.435969 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.435949    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.436422 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.436402    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.448143 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.448088    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vvk5\" (UniqueName: \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\") pod \"e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") " pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.526535 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.526511    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:12.807639 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:12.807607    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:12.809350 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:44:12.809319    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b637e66_6d62_4ebb_9b05_ba41ae742da7.slice/crio-52b76776da333a3070cc3ea84ed253f4b5c870c61bf6f450338493ff301e09c0 WatchSource:0}: Error finding container 52b76776da333a3070cc3ea84ed253f4b5c870c61bf6f450338493ff301e09c0: Status 404 returned error can't find the container with id 52b76776da333a3070cc3ea84ed253f4b5c870c61bf6f450338493ff301e09c0
Oct 30 19:44:13.532998 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:13.532965    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" event={"ID":"5b637e66-6d62-4ebb-9b05-ba41ae742da7","Type":"ContainerStarted","Data":"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"}
Oct 30 19:44:13.532998 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:13.533005    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" event={"ID":"5b637e66-6d62-4ebb-9b05-ba41ae742da7","Type":"ContainerStarted","Data":"52b76776da333a3070cc3ea84ed253f4b5c870c61bf6f450338493ff301e09c0"}
Oct 30 19:44:13.556586 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:13.556534    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" podStartSLOduration=1.55651692 podStartE2EDuration="1.55651692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:13.554981445 +0000 UTC m=+1518.524622282" watchObservedRunningTime="2026-10-30 19:44:13.55651692 +0000 UTC m=+1518.526157756"
Oct 30 19:44:14.526694 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:14.526626    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:16.238177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:16.237813    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-95d9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:16.238177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:16.237865    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-95d9v" podUID="f93f9c61-295c-4fe6-a44d-6611ed4631a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:16.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:16.263784    5033 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-8qdgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:16.276697 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:16.263883    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-8qdgp" podUID="d7dacba6-7983-4f68-8791-c46ed3bad49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:22.527063 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:22.527022    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:22.539688 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:22.539647    5033 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:22.592297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:22.592263    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:34.985868 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:44:34.985830    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:44:39.004040 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:39.003374    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:54.034477 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.034304    5033 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerName="main" containerID="cri-o://443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e" gracePeriod=45
Oct 30 19:44:54.640605 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.640579    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:54.744013 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.743978    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744035    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744063    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744094    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744177 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744152    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\" (UniqueName: \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744329 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744184    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\") pod \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\" (UID: \"5b637e66-6d62-4ebb-9b05-ba41ae742da7\") "
Oct 30 19:44:54.744465 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744434    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:54.744610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744588    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:54.744610 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.744600    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:54.746581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.746460    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:54.746581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.746460    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:54.746581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.746559    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5" pod "5b637e66-6d62-4ebb-9b05-ba41ae742da7" (UID: "5b637e66-6d62-4ebb-9b05-ba41ae742da7"). InnerVolumeSpecName "kube-api-access-5vvk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:54.832710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.832280    5033 generic.go:356] "Generic (PLEG): container finished" podID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerID="443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e" exitCode=0
Oct 30 19:44:54.832710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.832356    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" event={"ID":"5b637e66-6d62-4ebb-9b05-ba41ae742da7","Type":"ContainerDied","Data":"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"}
Oct 30 19:44:54.832710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.832395    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" event={"ID":"5b637e66-6d62-4ebb-9b05-ba41ae742da7","Type":"ContainerDied","Data":"52b76776da333a3070cc3ea84ed253f4b5c870c61bf6f450338493ff301e09c0"}
Oct 30 19:44:54.832710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.832422    5033 scope.go:122] "RemoveContainer" containerID="443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"
Oct 30 19:44:54.832710 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.832571    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"
Oct 30 19:44:54.844341 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.844320    5033 scope.go:122] "RemoveContainer" containerID="443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"
Oct 30 19:44:54.844648 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:44:54.844627    5033 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e\": container with ID starting with 443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e not found: ID does not exist" containerID="443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"
Oct 30 19:44:54.844730 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.844656    5033 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e"} err="failed to get container status \"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e\": rpc error: code = NotFound desc = could not find container \"443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e\": container with ID starting with 443ae3c4e78539f71c3f13d06ea6a1d51f678a7655787fcb05dac239fd1f924e not found: ID does not exist"
Oct 30 19:44:54.845029 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845010    5033 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-dshm\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.845067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845034    5033 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tmp-dir\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.845067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845043    5033 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-home\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.845067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845051    5033 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b637e66-6d62-4ebb-9b05-ba41ae742da7-model-cache\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.845067 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845060    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vvk5\" (UniqueName: \"kubernetes.io/projected/5b637e66-6d62-4ebb-9b05-ba41ae742da7-kube-api-access-5vvk5\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.845197 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.845070    5033 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b637e66-6d62-4ebb-9b05-ba41ae742da7-tls-certs\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.875902 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.875840    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:54.880759 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:54.880709    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j"]
Oct 30 19:44:55.195581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:55.195504    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" path="/var/lib/kubelet/pods/5b637e66-6d62-4ebb-9b05-ba41ae742da7/volumes"
Oct 30 19:44:55.587713 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:55.587660    5033 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-56e018f0-kserve-c749bf78f-zwm6j" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.137:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:56.429994 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:44:56.429952    5033 scope.go:122] "RemoveContainer" containerID="875abf5b21403adbe3e2ac8ff5bbb611b7a1e841a0c99aea6f294d85caf6c146"
Oct 30 19:45:00.168628 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.168593    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:00.169055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.169041    5033 cpu_manager.go:414] "containerMap: removing container" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerName="main"
Oct 30 19:45:00.169055 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.169055    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerName="main"
Oct 30 19:45:00.169129 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.169122    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b637e66-6d62-4ebb-9b05-ba41ae742da7" containerName="main"
Oct 30 19:45:00.173764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.173742    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.182166 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.182141    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:00.184471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.184448    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4kw2n\""
Oct 30 19:45:00.236100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.236071    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9sk5\" (UniqueName: \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.236240 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.236127    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.337642 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.337608    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9sk5\" (UniqueName: \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.337874 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.337694    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.338390 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.338369    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.347505 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.347478    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9sk5\" (UniqueName: \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\") pod \"maas-api-key-cleanup-29889825-s4gf9\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.489939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.489906    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:00.788637 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:45:00.788605    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c7b5952_2635_4265_bd73_29ad27315a3b.slice/crio-79df3f41bbc867747a2d3c5d0778f75c395e37cbb0be62c846485e48298b1bed WatchSource:0}: Error finding container 79df3f41bbc867747a2d3c5d0778f75c395e37cbb0be62c846485e48298b1bed: Status 404 returned error can't find the container with id 79df3f41bbc867747a2d3c5d0778f75c395e37cbb0be62c846485e48298b1bed
Oct 30 19:45:00.789597 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.789574    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:00.865190 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:00.865161    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerStarted","Data":"79df3f41bbc867747a2d3c5d0778f75c395e37cbb0be62c846485e48298b1bed"}
Oct 30 19:45:01.871764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:01.871699    5033 generic.go:356] "Generic (PLEG): container finished" podID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerID="14aa64c489a62881a735f624eb0709d2788417625a680bf33ab55a99b348146f" exitCode=60
Oct 30 19:45:01.871764 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:01.871746    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerDied","Data":"14aa64c489a62881a735f624eb0709d2788417625a680bf33ab55a99b348146f"}
Oct 30 19:45:01.872129 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:01.871989    5033 scope.go:122] "RemoveContainer" containerID="14aa64c489a62881a735f624eb0709d2788417625a680bf33ab55a99b348146f"
Oct 30 19:45:02.878898 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:02.878865    5033 generic.go:356] "Generic (PLEG): container finished" podID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerID="0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70" exitCode=60
Oct 30 19:45:02.879302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:02.878946    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerDied","Data":"0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70"}
Oct 30 19:45:02.879302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:02.879049    5033 scope.go:122] "RemoveContainer" containerID="14aa64c489a62881a735f624eb0709d2788417625a680bf33ab55a99b348146f"
Oct 30 19:45:02.879302 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:02.879272    5033 scope.go:122] "RemoveContainer" containerID="0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70"
Oct 30 19:45:02.879496 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:45:02.879477    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-s4gf9_odh-ai-gateway-infra(2c7b5952-2635-4265-bd73-29ad27315a3b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b"
Oct 30 19:45:03.885440 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:03.885411    5033 scope.go:122] "RemoveContainer" containerID="0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70"
Oct 30 19:45:03.885836 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:45:03.885575    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-s4gf9_odh-ai-gateway-infra(2c7b5952-2635-4265-bd73-29ad27315a3b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b"
Oct 30 19:45:04.985159 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:45:04.985127    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:45:15.164204 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:15.164116    5033 scope.go:122] "RemoveContainer" containerID="0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70"
Oct 30 19:45:15.951572 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:15.951531    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerStarted","Data":"aa618315e177ddaf9be96730a4a3884f024060512c71b6e2ed0fe96f1ca81b72"}
Oct 30 19:45:15.972091 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:15.972043    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" podStartSLOduration=15.972029227 podStartE2EDuration="15.972029227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:15.971271966 +0000 UTC m=+1580.940912949" watchObservedRunningTime="2026-10-30 19:45:15.972029227 +0000 UTC m=+1580.941670062"
Oct 30 19:45:16.958637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:16.958605    5033 generic.go:356] "Generic (PLEG): container finished" podID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerID="aa618315e177ddaf9be96730a4a3884f024060512c71b6e2ed0fe96f1ca81b72" exitCode=60
Oct 30 19:45:16.959103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:16.958703    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerDied","Data":"aa618315e177ddaf9be96730a4a3884f024060512c71b6e2ed0fe96f1ca81b72"}
Oct 30 19:45:16.959103 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:16.958781    5033 scope.go:122] "RemoveContainer" containerID="0e18ac96fc90248c4a2d02b4339a4d13214c8fe54da1a5b0630e40407b790b70"
Oct 30 19:45:16.959198 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:16.959105    5033 scope.go:122] "RemoveContainer" containerID="aa618315e177ddaf9be96730a4a3884f024060512c71b6e2ed0fe96f1ca81b72"
Oct 30 19:45:16.959359 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:45:16.959340    5033 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-s4gf9_odh-ai-gateway-infra(2c7b5952-2635-4265-bd73-29ad27315a3b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b"
Oct 30 19:45:16.985975 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:16.985946    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:18.093937 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.093914    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:18.181802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.181775    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\" (UniqueName: \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\") pod \"2c7b5952-2635-4265-bd73-29ad27315a3b\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") "
Oct 30 19:45:18.181944 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.181852    5033 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\") pod \"2c7b5952-2635-4265-bd73-29ad27315a3b\" (UID: \"2c7b5952-2635-4265-bd73-29ad27315a3b\") "
Oct 30 19:45:18.182174 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.182154    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca" pod "2c7b5952-2635-4265-bd73-29ad27315a3b" (UID: "2c7b5952-2635-4265-bd73-29ad27315a3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.183966 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.183942    5033 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5" pod "2c7b5952-2635-4265-bd73-29ad27315a3b" (UID: "2c7b5952-2635-4265-bd73-29ad27315a3b"). InnerVolumeSpecName "kube-api-access-b9sk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.283033 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.282974    5033 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c7b5952-2635-4265-bd73-29ad27315a3b-maas-api-service-ca\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.283134 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.283070    5033 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9sk5\" (UniqueName: \"kubernetes.io/projected/2c7b5952-2635-4265-bd73-29ad27315a3b-kube-api-access-b9sk5\") on node \"ip-10-0-143-135.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.971654 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.971624    5033 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"
Oct 30 19:45:18.971827 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.971653    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9" event={"ID":"2c7b5952-2635-4265-bd73-29ad27315a3b","Type":"ContainerDied","Data":"79df3f41bbc867747a2d3c5d0778f75c395e37cbb0be62c846485e48298b1bed"}
Oct 30 19:45:18.971827 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.971753    5033 scope.go:122] "RemoveContainer" containerID="aa618315e177ddaf9be96730a4a3884f024060512c71b6e2ed0fe96f1ca81b72"
Oct 30 19:45:18.997566 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:18.997539    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:19.001720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:19.001697    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-s4gf9"]
Oct 30 19:45:19.168050 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:19.168018    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" path="/var/lib/kubelet/pods/2c7b5952-2635-4265-bd73-29ad27315a3b/volumes"
Oct 30 19:45:23.314818 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.314780    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"]
Oct 30 19:45:23.315429 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315406    5033 cpu_manager.go:414] "containerMap: removing container" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315433    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315451    5033 cpu_manager.go:414] "containerMap: removing container" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315470 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315458    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315650    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315668    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.315720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.315691    5033 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:45:23.320637 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.320614    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.331165 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.330917    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:45:23.331371 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.331290    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-898sh\""
Oct 30 19:45:23.331471 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.331361    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 19:45:23.333457 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.333435    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"]
Oct 30 19:45:23.427386 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.427354    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5492f247-5f04-45d7-bc88-7948a5610c7f-openshift-service-ca\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.427525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.427398    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhhcz\" (UniqueName: \"kubernetes.io/projected/5492f247-5f04-45d7-bc88-7948a5610c7f-kube-api-access-bhhcz\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.427525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.427421    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-metrics-certs\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.427525 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.427466    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-maas-api-tls\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.528777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.528750    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-metrics-certs\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.528777 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.528779    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-maas-api-tls\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.528938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.528842    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5492f247-5f04-45d7-bc88-7948a5610c7f-openshift-service-ca\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.528938 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.528872    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhcz\" (UniqueName: \"kubernetes.io/projected/5492f247-5f04-45d7-bc88-7948a5610c7f-kube-api-access-bhhcz\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.529558 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.529536    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5492f247-5f04-45d7-bc88-7948a5610c7f-openshift-service-ca\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.531238 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.531215    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-maas-api-tls\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.531366 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.531348    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5492f247-5f04-45d7-bc88-7948a5610c7f-metrics-certs\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.543543 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.543524    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhhcz\" (UniqueName: \"kubernetes.io/projected/5492f247-5f04-45d7-bc88-7948a5610c7f-kube-api-access-bhhcz\") pod \"maas-api-5df7c6c68c-ngj2k\" (UID: \"5492f247-5f04-45d7-bc88-7948a5610c7f\") " pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.635345 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.635273    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:23.920662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.920333    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"]
Oct 30 19:45:23.922933 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:45:23.922894    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5492f247_5f04_45d7_bc88_7948a5610c7f.slice/crio-4ad632cca51d8dd33a59e25ede91bf1dd27a305f108cae4052d0d2ca35fcf619 WatchSource:0}: Error finding container 4ad632cca51d8dd33a59e25ede91bf1dd27a305f108cae4052d0d2ca35fcf619: Status 404 returned error can't find the container with id 4ad632cca51d8dd33a59e25ede91bf1dd27a305f108cae4052d0d2ca35fcf619
Oct 30 19:45:23.924334 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:23.924315    5033 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:45:24.000662 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:24.000634    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k" event={"ID":"5492f247-5f04-45d7-bc88-7948a5610c7f","Type":"ContainerStarted","Data":"4ad632cca51d8dd33a59e25ede91bf1dd27a305f108cae4052d0d2ca35fcf619"}
Oct 30 19:45:25.008438 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:25.008391    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k" event={"ID":"5492f247-5f04-45d7-bc88-7948a5610c7f","Type":"ContainerStarted","Data":"da04c01aa95f981180ac61f7fc00d9ce1b8ed733e8a8ce500c4eea00002612e5"}
Oct 30 19:45:25.008820 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:25.008591    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:25.032062 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:25.032014    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k" podStartSLOduration=1.6223521600000002 podStartE2EDuration="2.032002403s" totalImagesPullingTime="409.650243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:25.030125721 +0000 UTC m=+1589.999766747" watchObservedRunningTime="2026-10-30 19:45:25.032002403 +0000 UTC m=+1590.001643232"
Oct 30 19:45:31.021076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:45:31.021043    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df7c6c68c-ngj2k"
Oct 30 19:45:34.986215 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:45:34.986179    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:46:04.993371 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:46:04.993329    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:46:34.985978 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:46:34.985939    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:46:59.029305 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:46:59.029262    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"]
Oct 30 19:46:59.031352 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:46:59.031324    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-spwxg"]
Oct 30 19:46:59.167330 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:46:59.167302    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99d20ee7-0778-4cca-93ed-95e7c598f581" path="/var/lib/kubelet/pods/99d20ee7-0778-4cca-93ed-95e7c598f581/volumes"
Oct 30 19:47:04.985413 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:47:04.985375    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:47:19.028555 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:47:19.028519    5033 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"]
Oct 30 19:47:19.034027 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:47:19.034000    5033 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-mcnkq"]
Oct 30 19:47:19.168093 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:47:19.168066    5033 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c0f9d20-4476-4051-b9a2-3e3a945b9337" path="/var/lib/kubelet/pods/4c0f9d20-4476-4051-b9a2-3e3a945b9337/volumes"
Oct 30 19:47:34.985638 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:47:34.985603    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:48:04.987271 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:48:04.987226    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:48:34.986223 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:48:34.986188    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:48:55.445503 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:48:55.445470    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:48:55.450816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:48:55.450787    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:48:56.497095 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:48:56.497063    5033 scope.go:122] "RemoveContainer" containerID="b4b4dc603ea5a6a42b4714edaf29337a9b92ff14c94b01b250433cff4df70e91"
Oct 30 19:48:56.505663 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:48:56.505643    5033 scope.go:122] "RemoveContainer" containerID="d4674428148609449a5d2cd1de0791df5dcbbef850fc94bd348b1359c45ea19e"
Oct 30 19:49:03.641750 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:03.641718    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-pwkmf_891bf17a-d1a7-41dc-b318-df4c72b3ec05/manager/0.log"
Oct 30 19:49:04.038118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:04.038084    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-9mjnv_0e018cb3-2605-4a89-93d5-f27bc8522bf6/manager/0.log"
Oct 30 19:49:04.985853 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:49:04.985820    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:49:05.841016 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:05.840985    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df7c6c68c-ngj2k_5492f247-5f04-45d7-bc88-7948a5610c7f/maas-api/0.log"
Oct 30 19:49:07.117049 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:07.117016    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xddcz_ec0512be-25bd-486b-9310-54f908948157/postgres/0.log"
Oct 30 19:49:08.868452 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:08.868420    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sxzhz_13ae1ec1-cef4-4a7a-aadf-4197b010f02e/manager/0.log"
Oct 30 19:49:09.071382 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:09.071357    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-sr2jh_ab0b4b4e-4979-40ac-9be8-3864e5ab9628/manager/0.log"
Oct 30 19:49:09.453193 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:09.453166    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-qwtdd_b68ff9f8-167b-4de0-9122-7b30984755c8/manager/0.log"
Oct 30 19:49:10.776237 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:10.776204    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b_aba00d88-8a0d-48ce-a232-ee2521c70bb9/istio-proxy/0.log"
Oct 30 19:49:11.516967 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:11.516934    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-8qdgp_d7dacba6-7983-4f68-8791-c46ed3bad49f/istio-proxy/0.log"
Oct 30 19:49:11.695702 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:11.695662    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-95d9v_f93f9c61-295c-4fe6-a44d-6611ed4631a7/istio-proxy/0.log"
Oct 30 19:49:12.774326 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:12.774298    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-sx9vv_f0f928a1-6d04-4493-82ff-d83fda077a36/istio-proxy/0.log"
Oct 30 19:49:13.388317 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:13.388281    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g_c6e0d197-586c-47f8-a13d-693cd930ac03/istio-proxy/0.log"
Oct 30 19:49:14.004267 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:14.004238    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"
Oct 30 19:49:26.912004 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:26.911966    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-pwkmf_891bf17a-d1a7-41dc-b318-df4c72b3ec05/manager/0.log"
Oct 30 19:49:27.338993 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:27.338958    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-9mjnv_0e018cb3-2605-4a89-93d5-f27bc8522bf6/manager/0.log"
Oct 30 19:49:28.886205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:28.886162    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df7c6c68c-ngj2k_5492f247-5f04-45d7-bc88-7948a5610c7f/maas-api/0.log"
Oct 30 19:49:30.197952 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:30.197919    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xddcz_ec0512be-25bd-486b-9310-54f908948157/postgres/0.log"
Oct 30 19:49:31.604552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:31.604522    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sxzhz_13ae1ec1-cef4-4a7a-aadf-4197b010f02e/manager/0.log"
Oct 30 19:49:31.794613 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:31.794582    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-sr2jh_ab0b4b4e-4979-40ac-9be8-3864e5ab9628/manager/0.log"
Oct 30 19:49:32.166705 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:32.166668    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-qwtdd_b68ff9f8-167b-4de0-9122-7b30984755c8/manager/0.log"
Oct 30 19:49:33.129636 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:33.129587    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b_aba00d88-8a0d-48ce-a232-ee2521c70bb9/istio-proxy/0.log"
Oct 30 19:49:33.845458 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:33.845422    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-8qdgp_d7dacba6-7983-4f68-8791-c46ed3bad49f/istio-proxy/0.log"
Oct 30 19:49:34.037433 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:34.037405    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-95d9v_f93f9c61-295c-4fe6-a44d-6611ed4631a7/istio-proxy/0.log"
Oct 30 19:49:34.986792 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:49:34.986751    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:49:35.124521 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:35.124488    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-sx9vv_f0f928a1-6d04-4493-82ff-d83fda077a36/istio-proxy/0.log"
Oct 30 19:49:35.661901 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:35.661787    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g_c6e0d197-586c-47f8-a13d-693cd930ac03/istio-proxy/0.log"
Oct 30 19:49:36.220273 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:49:36.220240    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"
Oct 30 19:50:04.986992 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:50:04.986951    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:50:34.989116 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:50:34.989072    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:51:04.985351 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:51:04.985312    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:51:09.523205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:09.523092    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5qbqv_03f7a52b-5c76-43a0-8310-3c56ce2e07d2/global-pull-secret-syncer/0.log"
Oct 30 19:51:09.650076 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:09.650046    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-67jwx_52a1b4f2-94a1-4c37-b905-296e9d8e1513/connection-checker/0.log"
Oct 30 19:51:09.677123 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:09.677098    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-897tx_393c0959-53ac-4bb4-b156-908f228ce12e/connection-checker/0.log"
Oct 30 19:51:09.705340 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:09.705318    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-9j7fn_d976ba7e-0594-4893-9ef9-4332e38a27c8/connection-checker/0.log"
Oct 30 19:51:09.855326 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:09.855249    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s2mwq_dc1db0ad-000a-4196-a19f-d452c4856568/konnectivity-agent/0.log"
Oct 30 19:51:10.019360 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:10.019333    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-135.us-east-2.compute.internal_c51f119163141e2efcccd6dec5fb66e6/haproxy/0.log"
Oct 30 19:51:17.147862 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:17.147829    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-sxzhz_13ae1ec1-cef4-4a7a-aadf-4197b010f02e/manager/0.log"
Oct 30 19:51:17.185746 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:17.185727    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-sr2jh_ab0b4b4e-4979-40ac-9be8-3864e5ab9628/manager/0.log"
Oct 30 19:51:17.282059 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:17.282031    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-qwtdd_b68ff9f8-167b-4de0-9122-7b30984755c8/manager/0.log"
Oct 30 19:51:18.836789 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:18.836754    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-g24tb_f02c732c-d967-4721-acf7-76933fd1aba1/cluster-monitoring-operator/0.log"
Oct 30 19:51:19.230736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.230708    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkr48_d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2/node-exporter/0.log"
Oct 30 19:51:19.263843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.263816    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkr48_d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2/kube-rbac-proxy/0.log"
Oct 30 19:51:19.291312 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.291294    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkr48_d9dd5ae6-4dfc-4948-a4bd-6c2b65a733d2/init-textfile/0.log"
Oct 30 19:51:19.367780 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.367752    5033 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"]
Oct 30 19:51:19.368843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.368817    5033 cpu_manager.go:414] "containerMap: removing container" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:51:19.368843 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.368839    5033 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c7b5952-2635-4265-bd73-29ad27315a3b" containerName="cleanup"
Oct 30 19:51:19.377434 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.377413    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.380826 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.380798    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"]
Oct 30 19:51:19.404393 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.404374    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-lib-modules\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.404500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.404407    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls4zg\" (UniqueName: \"kubernetes.io/projected/b5dff568-a1c3-4b65-8156-91619198fa96-kube-api-access-ls4zg\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.404500 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.404443    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-proc\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.404618 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.404596    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-sys\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.404661 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.404652    5033 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-podres\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.464882 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.464860    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8mgz\"/\"openshift-service-ca.crt\""
Oct 30 19:51:19.464975 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.464911    5033 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d8mgz\"/\"default-dockercfg-8p9jx\""
Oct 30 19:51:19.465020 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.464976    5033 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8mgz\"/\"kube-root-ca.crt\""
Oct 30 19:51:19.505788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505738    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-sys\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505788 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505769    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-podres\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505807    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-lib-modules\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505830    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls4zg\" (UniqueName: \"kubernetes.io/projected/b5dff568-a1c3-4b65-8156-91619198fa96-kube-api-access-ls4zg\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505852    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-sys\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505858    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-podres\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505876    5033 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-proc\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.505939 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505903    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-proc\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.506158 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.505957    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5dff568-a1c3-4b65-8156-91619198fa96-lib-modules\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.515666 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.515646    5033 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls4zg\" (UniqueName: \"kubernetes.io/projected/b5dff568-a1c3-4b65-8156-91619198fa96-kube-api-access-ls4zg\") pod \"perf-node-gather-daemonset-mgx9v\" (UID: \"b5dff568-a1c3-4b65-8156-91619198fa96\") " pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.692552 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.692524    5033 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:19.729857 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.729832    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-27fpf_b109aa02-9164-47d5-8c6a-704935fea276/prometheus-operator-admission-webhook/0.log"
Oct 30 19:51:19.978205 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:19.978177    5033 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"]
Oct 30 19:51:19.979379 ip-10-0-143-135 kubenswrapper[5033]: W1030 19:51:19.979352    5033 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5dff568_a1c3_4b65_8156_91619198fa96.slice/crio-ee5862005a9c2bd9a084f01bc1c487ad1c8e012c2379afd96eb4771d524d180a WatchSource:0}: Error finding container ee5862005a9c2bd9a084f01bc1c487ad1c8e012c2379afd96eb4771d524d180a: Status 404 returned error can't find the container with id ee5862005a9c2bd9a084f01bc1c487ad1c8e012c2379afd96eb4771d524d180a
Oct 30 19:51:20.859100 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:20.859064    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v" event={"ID":"b5dff568-a1c3-4b65-8156-91619198fa96","Type":"ContainerStarted","Data":"ceea089df6cecfa48da7c797be510fc613f274d529a0f3e4d2e725988699db3f"}
Oct 30 19:51:20.859257 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:20.859214    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:20.859297 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:20.859262    5033 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v" event={"ID":"b5dff568-a1c3-4b65-8156-91619198fa96","Type":"ContainerStarted","Data":"ee5862005a9c2bd9a084f01bc1c487ad1c8e012c2379afd96eb4771d524d180a"}
Oct 30 19:51:20.879587 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:20.879543    5033 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v" podStartSLOduration=1.879531149 podStartE2EDuration="1.879531149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:20.877405118 +0000 UTC m=+1945.847045965" watchObservedRunningTime="2026-10-30 19:51:20.879531149 +0000 UTC m=+1945.849171984"
Oct 30 19:51:21.252804 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:21.252777    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-wtwxl_962ae636-bcad-4264-9b99-b47896c634e1/networking-console-plugin/0.log"
Oct 30 19:51:21.937163 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:21.937137    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/1.log"
Oct 30 19:51:21.941947 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:21.941925    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b7c9l_cf2c3f45-829b-4784-b1da-5997daf80443/console-operator/2.log"
Oct 30 19:51:22.603581 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:22.603552    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-njm2t_cadcee6d-8c2f-46ad-8f21-8adb13874d33/download-server/0.log"
Oct 30 19:51:23.268664 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:23.268636    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-g46gd_b509b764-b249-45ca-8c13-38332fc6c486/volume-data-source-validator/0.log"
Oct 30 19:51:24.631117 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:24.631086    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmrtr_fe0bce47-ad9b-4cbe-b544-30d75ae93300/dns/0.log"
Oct 30 19:51:24.656441 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:24.656421    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmrtr_fe0bce47-ad9b-4cbe-b544-30d75ae93300/kube-rbac-proxy/0.log"
Oct 30 19:51:24.747802 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:24.747785    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ln7fq_07071eae-8f8c-4d40-a3c3-57db59367dfb/dns-node-resolver/0.log"
Oct 30 19:51:25.388052 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:25.388021    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7dc79d54f-wjdcb_49b322c5-f1df-4b2f-bf50-e8f6dfc46493/registry/0.log"
Oct 30 19:51:25.412138 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:25.412115    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-85l8w_5895a27a-6712-4344-ac33-f49f30f5e920/node-ca/0.log"
Oct 30 19:51:26.818720 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:26.818671    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b_aba00d88-8a0d-48ce-a232-ee2521c70bb9/istio-proxy/0.log"
Oct 30 19:51:26.875075 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:26.875052    5033 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d8mgz/perf-node-gather-daemonset-mgx9v"
Oct 30 19:51:28.202736 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:28.202707    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-8qdgp_d7dacba6-7983-4f68-8791-c46ed3bad49f/istio-proxy/0.log"
Oct 30 19:51:28.236374 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:28.236349    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-95d9v_f93f9c61-295c-4fe6-a44d-6611ed4631a7/istio-proxy/0.log"
Oct 30 19:51:28.452978 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:28.452912    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-sx9vv_f0f928a1-6d04-4493-82ff-d83fda077a36/istio-proxy/0.log"
Oct 30 19:51:28.576575 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:28.576550    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g_c6e0d197-586c-47f8-a13d-693cd930ac03/istio-proxy/0.log"
Oct 30 19:51:28.682509 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:28.682488    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"
Oct 30 19:51:30.048434 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:30.048407    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b_aba00d88-8a0d-48ce-a232-ee2521c70bb9/istio-proxy/0.log"
Oct 30 19:51:30.395186 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:30.395118    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pt2v2_cbec4229-cce9-465d-8ff4-c01460c9a1b9/serve-healthcheck-canary/0.log"
Oct 30 19:51:31.312037 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.312008    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-c8m4q_cda74375-ee6c-4daa-a5dd-74d0df935567/insights-operator/0.log"
Oct 30 19:51:31.312469 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.312208    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-c8m4q_cda74375-ee6c-4daa-a5dd-74d0df935567/insights-operator/1.log"
Oct 30 19:51:31.344829 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.344808    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-8qdgp_d7dacba6-7983-4f68-8791-c46ed3bad49f/istio-proxy/0.log"
Oct 30 19:51:31.359202 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.359186    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6rzj_12011250-d65c-4fa7-9529-bd4731595e41/kube-rbac-proxy/0.log"
Oct 30 19:51:31.369772 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.369757    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-95d9v_f93f9c61-295c-4fe6-a44d-6611ed4631a7/istio-proxy/0.log"
Oct 30 19:51:31.387940 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.387922    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6rzj_12011250-d65c-4fa7-9529-bd4731595e41/exporter/0.log"
Oct 30 19:51:31.430354 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.430335    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6rzj_12011250-d65c-4fa7-9529-bd4731595e41/extractor/0.log"
Oct 30 19:51:31.536420 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.536402    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-sx9vv_f0f928a1-6d04-4493-82ff-d83fda077a36/istio-proxy/0.log"
Oct 30 19:51:31.640242 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.640189    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g_c6e0d197-586c-47f8-a13d-693cd930ac03/istio-proxy/0.log"
Oct 30 19:51:31.749568 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.749550    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"
Oct 30 19:51:31.821291 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:31.821270    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-fkvb6-58b59_a9039dbb-e81d-432b-8b47-0c51777dbc6f/insights-gathering/0.log"
Oct 30 19:51:33.694508 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:33.694473    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57d8447fc-gvh2b_aba00d88-8a0d-48ce-a232-ee2521c70bb9/istio-proxy/0.log"
Oct 30 19:51:34.693667 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:34.693640    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-668f68f55d-sbzvc_c50d5442-fff2-47ab-8ee9-a6375c2b46df/manager/0.log"
Oct 30 19:51:34.727056 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:34.727038    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-668f68f55d-sbzvc_c50d5442-fff2-47ab-8ee9-a6375c2b46df/copy-manifests/0.log"
Oct 30 19:51:34.985748 ip-10-0-143-135 kubenswrapper[5033]: E1030 19:51:34.985715    5033 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6rzj" containerName="extractor"
Oct 30 19:51:35.020782 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:35.020761    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-8qdgp_d7dacba6-7983-4f68-8791-c46ed3bad49f/istio-proxy/0.log"
Oct 30 19:51:35.038816 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:35.038798    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-95d9v_f93f9c61-295c-4fe6-a44d-6611ed4631a7/istio-proxy/0.log"
Oct 30 19:51:35.172819 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:35.172800    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-sx9vv_f0f928a1-6d04-4493-82ff-d83fda077a36/istio-proxy/0.log"
Oct 30 19:51:35.227118 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:35.227080    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-86dbb48f59-qjx4g_c6e0d197-586c-47f8-a13d-693cd930ac03/istio-proxy/0.log"
Oct 30 19:51:35.296516 ip-10-0-143-135 kubenswrapper[5033]: I1030 19:51:35.296455    5033 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5c44fb6c-bc7gp_72965977-109a-4e75-b0f5-31f94e98c42a/router/0.log"